{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction\n",
    "\n",
    "This notebook outlines how to explain the results of a recommendation system built using a Factorization Machine (FM) model in Amazon SageMaker.\n",
    "\n",
    "There are four parts to this notebook:\n",
    "\n",
    "1. Building a FM Model\n",
    "2. Extracting FM model parameters\n",
    "3. Building the influence matrix\n",
    "4. Explaining recommendations for a user\n",
    "\n",
    "## Data sources and citations\n",
    "\n",
    "I used three primary sources for this notebook.\n",
    "\n",
    "### How to build and extract FM model\n",
    "\n",
    "[Extending Amazon SageMaker factorization machines algorithm to predict top x recommendations](https://aws.amazon.com/blogs/machine-learning/extending-amazon-sagemaker-factorization-machines-algorithm-to-predict-top-x-recommendations/), published on the AWS Machine Learning Blog by Zohar Karnin and Rama Thamman on April 5, 2019.  This blog has a sample notebook for building the FM model for the movie lens dataset and extracting the FM model parameters.  I am repeating Parts 1 and 2 from that notebook for the sake of being able to reproduce the entire workflow in a single notebook.\n",
    "\n",
    "### How to build influence matrix\n",
    "\n",
    "I implemented the technique in this paper for building the influence matrix:\n",
    "\n",
    "Bashir Rastegarpanah, Mark Crovella, Krishna Gummadi. 2017. \"Exploring Explanations for Matrix Factorization Recommender Systems (Position Paper).\" Proceedings of the FATREC Workshop on Responsible Recommendation.  Retrieved on October 7, 2019, from https://hdl.handle.net/2144/26683.\n",
    "\n",
    "\n",
    "### Data set\n",
    "\n",
    "The blog published by Karnin and Thamman uses the GroupLens movie dataset, available on https://grouplens.org/datasets/movielens/.  Per the terms of reuse, we do not redistribute the data set here, but rather provide code to download it.  The dataset formal citation is:\n",
    "\n",
    "F. Maxwell Harper and Joseph A. Konstan. 2015. The MovieLens Datasets: History and Context. ACM Transactions on Interactive Intelligent Systems (TiiS) 5, 4, Article 19 (December 2015), 19 pages. DOI=http://dx.doi.org/10.1145/2827872"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 1 - Building a FM Model using movie lens dataset\n",
    "\n",
    "This section is reproduced with minor modifications from the blog cited above.  I include it for completeness so you can see how to build the FM model from the source data set.\n",
    "\n",
    "Be sure to customize the name of the S3 bucket used to upload the data set for FM training.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import boto3\n",
    "import sagemaker\n",
    "from sagemaker import get_execution_role\n",
    "\n",
    "sagemaker_session = sagemaker.Session()\n",
    "role = get_execution_role()\n",
    "bucket = sagemaker_session.default_bucket()\n",
    "region_name = boto3.Session().region_name\n",
    "\n",
    "data_prefix = 'sagemaker/factorization-machines/movielens/data'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import sagemaker\n",
    "import sagemaker.amazon.common as smac\n",
    "from sagemaker import get_execution_role\n",
    "from sagemaker.predictor import json_deserializer\n",
    "from sagemaker.amazon.amazon_estimator import get_image_uri\n",
    "import numpy as np\n",
    "from scipy.sparse import lil_matrix\n",
    "import pandas as pd\n",
    "import boto3, io, os"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Download movie rating data from movie lens"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--2020-11-08 21:37:08--  http://files.grouplens.org/datasets/movielens/ml-100k.zip\n",
      "Resolving files.grouplens.org (files.grouplens.org)... 128.101.65.152\n",
      "Connecting to files.grouplens.org (files.grouplens.org)|128.101.65.152|:80... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: 4924029 (4.7M) [application/zip]\n",
      "Saving to: ‘ml-100k.zip.4’\n",
      "\n",
      "ml-100k.zip.4       100%[===================>]   4.70M  13.9MB/s    in 0.3s    \n",
      "\n",
      "2020-11-08 21:37:08 (13.9 MB/s) - ‘ml-100k.zip.4’ saved [4924029/4924029]\n",
      "\n",
      "Archive:  ml-100k.zip\n",
      "  inflating: ml-100k/allbut.pl       \n",
      "  inflating: ml-100k/mku.sh          \n",
      "  inflating: ml-100k/README          \n",
      "  inflating: ml-100k/u.data          \n",
      "  inflating: ml-100k/u.genre         \n",
      "  inflating: ml-100k/u.info          \n",
      "  inflating: ml-100k/u.item          \n",
      "  inflating: ml-100k/u.occupation    \n",
      "  inflating: ml-100k/u.user          \n",
      "  inflating: ml-100k/u1.base         \n",
      "  inflating: ml-100k/u1.test         \n",
      "  inflating: ml-100k/u2.base         \n",
      "  inflating: ml-100k/u2.test         \n",
      "  inflating: ml-100k/u3.base         \n",
      "  inflating: ml-100k/u3.test         \n",
      "  inflating: ml-100k/u4.base         \n",
      "  inflating: ml-100k/u4.test         \n",
      "  inflating: ml-100k/u5.base         \n",
      "  inflating: ml-100k/u5.test         \n",
      "  inflating: ml-100k/ua.base         \n",
      "  inflating: ml-100k/ua.test         \n",
      "  inflating: ml-100k/ub.base         \n",
      "  inflating: ml-100k/ub.test         \n"
     ]
    }
   ],
   "source": [
    "#download data\n",
    "!wget http://files.grouplens.org/datasets/movielens/ml-100k.zip\n",
    "!unzip -o ml-100k.zip"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Shuffle the data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "!shuf ml-100k/ua.base -o ml-100k/ua.base.shuffled"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Load Training Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>user_id</th>\n",
       "      <th>movie_id</th>\n",
       "      <th>rating</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>756</td>\n",
       "      <td>258</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>795</td>\n",
       "      <td>151</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>392</td>\n",
       "      <td>272</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>661</td>\n",
       "      <td>433</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>234</td>\n",
       "      <td>483</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   user_id  movie_id  rating\n",
       "0      756       258       3\n",
       "1      795       151       3\n",
       "2      392       272       5\n",
       "3      661       433       5\n",
       "4      234       483       5"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "user_movie_ratings_train = pd.read_csv('ml-100k/ua.base.shuffled', sep='\\t', index_col=False, \n",
    "                 names=['user_id' , 'movie_id' , 'rating'])\n",
    "user_movie_ratings_train.head(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Load Test Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>user_id</th>\n",
       "      <th>movie_id</th>\n",
       "      <th>rating</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>20</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>33</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>61</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>117</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>155</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   user_id  movie_id  rating\n",
       "0        1        20       4\n",
       "1        1        33       4\n",
       "2        1        61       4\n",
       "3        1       117       3\n",
       "4        1       155       2"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "user_movie_ratings_test = pd.read_csv('ml-100k/ua.test', sep='\\t', index_col=False, \n",
    "                 names=['user_id' , 'movie_id' , 'rating'])\n",
    "user_movie_ratings_test.head(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " # of users:  943\n",
      " # of movies:  1682\n",
      " Training Count:  90570\n",
      " Test Count:  9430\n",
      " Features (# of users + # of movies):  2625\n"
     ]
    }
   ],
   "source": [
    "nb_users= user_movie_ratings_train['user_id'].max()\n",
    "nb_movies=user_movie_ratings_train['movie_id'].max()\n",
    "nb_features=nb_users+nb_movies\n",
    "nb_ratings_test=len(user_movie_ratings_test.index)\n",
    "nb_ratings_train=len(user_movie_ratings_train.index)\n",
    "print(\" # of users: \", nb_users)\n",
    "print(\" # of movies: \", nb_movies)\n",
    "print(\" Training Count: \", nb_ratings_train)\n",
    "print(\" Test Count: \", nb_ratings_test)\n",
    "print(\" Features (# of users + # of movies): \", nb_features)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### FM Input\n",
    "\n",
    "Input to FM is a one-hot encoded sparse matrix. Only ratings 4 and above are considered for the model. We will be ignoring ratings 3 and below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "def loadDataset(df, lines, columns):\n",
    "    # Features are one-hot encoded in a sparse matrix\n",
    "    X = lil_matrix((lines, columns)).astype('float32')\n",
    "    # Labels are stored in a vector\n",
    "    Y = []\n",
    "    line=0\n",
    "    for index, row in df.iterrows():\n",
    "            X[line,row['user_id']-1] = 1\n",
    "            X[line, nb_users+(row['movie_id']-1)] = 1\n",
    "            if int(row['rating']) >= 4:\n",
    "                Y.append(1)\n",
    "            else:\n",
    "                Y.append(0)\n",
    "            line=line+1\n",
    "\n",
    "    Y=np.array(Y).astype('float32')            \n",
    "    return X,Y\n",
    "\n",
    "\n",
    "X_train, Y_train = loadDataset(user_movie_ratings_train, nb_ratings_train, nb_features)\n",
    "X_test, Y_test = loadDataset(user_movie_ratings_test, nb_ratings_test, nb_features)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(90570, 2625)\n",
      "(90570,)\n",
      "Training labels: 49906 zeros, 40664 ones\n",
      "(9430, 2625)\n",
      "(9430,)\n",
      "Test labels: 5469 zeros, 3961 ones\n"
     ]
    }
   ],
   "source": [
    "print(X_train.shape)\n",
    "print(Y_train.shape)\n",
    "assert X_train.shape == (nb_ratings_train, nb_features)\n",
    "assert Y_train.shape == (nb_ratings_train, )\n",
    "zero_labels = np.count_nonzero(Y_train)\n",
    "print(\"Training labels: %d zeros, %d ones\" % (zero_labels, nb_ratings_train-zero_labels))\n",
    "\n",
    "print(X_test.shape)\n",
    "print(Y_test.shape)\n",
    "assert X_test.shape  == (nb_ratings_test, nb_features)\n",
    "assert Y_test.shape  == (nb_ratings_test, )\n",
    "zero_labels = np.count_nonzero(Y_test)\n",
    "print(\"Test labels: %d zeros, %d ones\" % (zero_labels, nb_ratings_test-zero_labels))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Convert to Protobuf format for saving to S3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "prefix = 'fm'\n",
    "\n",
    "if bucket.strip() == '':\n",
    "    raise RuntimeError(\"bucket name is empty.\")\n",
    "\n",
    "train_key      = 'train.protobuf'\n",
    "train_prefix   = '{}/{}'.format(prefix, 'train')\n",
    "\n",
    "test_key       = 'test.protobuf'\n",
    "test_prefix    = '{}/{}'.format(prefix, 'test')\n",
    "\n",
    "output_prefix  = 's3://{}/{}/output'.format(bucket, prefix)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training data S3 path:  s3://sagemaker-us-east-1-835319576252/fm/train/train.protobuf\n",
      "Test data S3 path:  s3://sagemaker-us-east-1-835319576252/fm/test/test.protobuf\n",
      "FM model output S3 path: s3://sagemaker-us-east-1-835319576252/fm/output\n"
     ]
    }
   ],
   "source": [
    "def writeDatasetToProtobuf(X, bucket, prefix, key, d_type, Y=None):\n",
    "    buf = io.BytesIO()\n",
    "    if d_type == \"sparse\":\n",
    "        smac.write_spmatrix_to_sparse_tensor(buf, X, labels=Y)\n",
    "    else:\n",
    "        smac.write_numpy_to_dense_tensor(buf, X, labels=Y)\n",
    "        \n",
    "    buf.seek(0)\n",
    "    obj = '{}/{}'.format(prefix, key)\n",
    "    boto3.resource('s3').Bucket(bucket).Object(obj).upload_fileobj(buf)\n",
    "    return 's3://{}/{}'.format(bucket,obj)\n",
    "    \n",
    "fm_train_data_path = writeDatasetToProtobuf(X_train, bucket, train_prefix, train_key, \"sparse\", Y_train)    \n",
    "fm_test_data_path  = writeDatasetToProtobuf(X_test, bucket, test_prefix, test_key, \"sparse\", Y_test)    \n",
    "  \n",
    "print(\"Training data S3 path: \",fm_train_data_path)\n",
    "print(\"Test data S3 path: \",fm_test_data_path)\n",
    "print(\"FM model output S3 path: {}\".format(output_prefix))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Run training job\n",
    "\n",
    "You can play around with the hyper parameters until you are happy with the prediction. For this dataset and hyper parameters configuration, after 200 epochs, test accuracy was around 70% on average and the F1 score (a typical metric for a binary classifier) was around 0.75 (1 indicates a perfect classifier). Not great, but you can fine tune the model further."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you've already run the training job, you can load it rather than running the job again.  Just set the `model_uri` parameter to the location of the model artifact, and set the flag `model_exists` to `True`.\n",
    "\n",
    "Similarly, if you already have a prediction endpoint available, set the flag `model_deployed` to `True` and provide the `model_endpoint` parameter."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "# # https://github.com/aws/sagemaker-python-sdk/issues/1985\n",
    "# # container = sagemaker.image_uris.retrieve(region_name, \"blazingtext\", \"latest\")\n",
    "\n",
    "# image_uri = ''\n",
    "\n",
    "# if region_name == 'us-west-1':\n",
    "#     image_uri = '632365934929.dkr.ecr.us-west-1.amazonaws.com'\n",
    "\n",
    "# if region_name == 'us-west-2':\n",
    "#     image_uri = '433757028032.dkr.ecr.us-west-2.amazonaws.com'\n",
    "    \n",
    "# if region_name =='us-east-1':\n",
    "#     image_uri = '811284229777.dkr.ecr.us-east-1.amazonaws.com'\n",
    "\n",
    "# if region_name == 'us-east-2':\n",
    "#     image_uri = '825641698319.dkr.ecr.us-east-2.amazonaws.com'\n",
    "\n",
    "# if region_name =='ap-east-1':\n",
    "#     image_uri = '286214385809.dkr.ecr.ap-east-1.amazonaws.com'\n",
    "\n",
    "# if region_name == 'ap-northeast-1':\n",
    "#     image_uri = '501404015308.dkr.ecr.ap-northeast-1.amazonaws.com'\n",
    "\n",
    "# if region_name == 'ap-northeast-2':\n",
    "#     image_uri = '306986355934.dkr.ecr.ap-northeast-2.amazonaws.com'\n",
    "\n",
    "# if region_name == 'ap-south-1':\n",
    "#     image_uri = '991648021394.dkr.ecr.ap-south-1.amazonaws.com'\n",
    "\n",
    "# if region_name == 'ap-southeast-1':\n",
    "#     image_uri = '475088953585.dkr.ecr.ap-southeast-1.amazonaws.com'\n",
    "\n",
    "# if region_name == 'ap-southeast-2':\n",
    "#     image_uri = '544295431143.dkr.ecr.ap-southeast-2.amazonaws.com'\n",
    "\n",
    "# if region_name == 'ca-central-1':\n",
    "#     image_uri = '469771592824.dkr.ecr.ca-central-1.amazonaws.com'\n",
    "\n",
    "# if region_name == 'cn-north-1':\n",
    "#     image_uri = '390948362332.dkr.ecr.cn-north-1.amazonaws.com.cn'\n",
    "\n",
    "# if region_name == 'cn-northwest-1':\n",
    "#     image_uri = '387376663083.dkr.ecr.cn-northwest-1.amazonaws.com.cn'\n",
    "\n",
    "# if region_name == 'eu-central-1': \n",
    "#     image_uri = '813361260812.dkr.ecr.eu-central-1.amazonaws.com'\n",
    "\n",
    "# if region_name == 'eu-north-1':\n",
    "#     image_uri = '669576153137.dkr.ecr.eu-north-1.amazonaws.com'\n",
    "\n",
    "# if region_name == 'eu-west-1':\n",
    "#     image_uri = '685385470294.dkr.ecr.eu-west-1.amazonaws.com'\n",
    "\n",
    "# if region_name == 'eu-west-2':\n",
    "#     image_uri = '644912444149.dkr.ecr.eu-west-2.amazonaws.com'\n",
    "\n",
    "# if region_name == 'eu-west-3':\n",
    "#     image_uri = '749696950732.dkr.ecr.eu-west-3.amazonaws.com'\n",
    "\n",
    "# if region_name == 'me-south-1':\n",
    "#     image_uri = '249704162688.dkr.ecr.me-south-1.amazonaws.com'\n",
    "    \n",
    "# if region_name == 'sa-east-1':\n",
    "#     image_uri = '855470959533.dkr.ecr.sa-east-1.amazonaws.com'\n",
    "\n",
    "# if region_name == 'us-gov-west-1':\n",
    "#     image_uri = '226302683700.dkr.ecr.us-gov-west-1.amazonaws.com'\n",
    "    \n",
    "# # https://docs.aws.amazon.com/en_us/sagemaker/latest/dg/sagemaker-algo-docker-registry-paths.html\n",
    "# image_uri = '{}/factorization-machines:1'.format(image_uri)\n",
    "\n",
    "# print('Using SageMaker container: {} ({})'.format(image_uri, region_name))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "# image_uri = get_image_uri(region_name, \"factorization-machines\")\n",
    "# print(image_uri)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "image_uri = '382416733822.dkr.ecr.us-east-1.amazonaws.com/factorization-machines:1'\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "model_exists = True\n",
    "model_uri = 's3://{}/factorization-machines/movielens/output/factorization-machines-2019-10-10-22-13-15-602/output/model.tar.gz'.format(bucket)\n",
    "model_deployed = True\n",
    "#model_endpoint = 'factorization-machines-2019-10-11-15-27-16-815' \n",
    "#if model_exists:\n",
    "#    fm_model = sagemaker.FactorizationMachinesModel(model_uri, get_execution_role(), sagemaker_session=sagemaker.Session())\n",
    "    \n",
    "#    if model_deployed:\n",
    "#        fm_predictor =  sagemaker.predictor.RealTimePredictor(model_endpoint, sagemaker_session=sagemaker.Session())\n",
    "#    else:\n",
    "#        fm_predictor = fm_model.deploy(initial_instance_count=1,\n",
    "#                         instance_type='ml.m5.xlarge')\n",
    "#else:\n",
    "fm = sagemaker.estimator.Estimator(image_uri=image_uri,\n",
    "                                   role=role, \n",
    "                                   instance_count=1, \n",
    "                                   instance_type='ml.m5.xlarge',\n",
    "                                   output_path=output_prefix,\n",
    "                                   sagemaker_session=sagemaker.Session())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2020-11-08 21:37:30 Starting - Starting the training job...\n",
      "2020-11-08 21:37:32 Starting - Launching requested ML instances......\n",
      "2020-11-08 21:38:43 Starting - Preparing the instances for training...\n",
      "2020-11-08 21:39:28 Downloading - Downloading input data...\n",
      "2020-11-08 21:39:52 Training - Downloading the training image..\u001b[34mDocker entrypoint called with argument(s): train\u001b[0m\n",
      "\u001b[34mRunning default environment configuration script\u001b[0m\n",
      "\u001b[34m/opt/amazon/lib/python2.7/site-packages/pandas/util/nosetester.py:13: DeprecationWarning: Importing from numpy.testing.nosetester is deprecated, import from numpy.testing instead.\n",
      "  from numpy.testing import nosetester\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:09 INFO 139813082523456] Reading default configuration from /opt/amazon/lib/python2.7/site-packages/algorithm/resources/default-conf.json: {u'factors_lr': u'0.0001', u'linear_init_sigma': u'0.01', u'epochs': 1, u'_wd': u'1.0', u'_num_kv_servers': u'auto', u'use_bias': u'true', u'factors_init_sigma': u'0.001', u'_log_level': u'info', u'bias_init_method': u'normal', u'linear_init_method': u'normal', u'linear_lr': u'0.001', u'factors_init_method': u'normal', u'_tuning_objective_metric': u'', u'bias_wd': u'0.01', u'use_linear': u'true', u'bias_lr': u'0.1', u'mini_batch_size': u'1000', u'_use_full_symbolic': u'true', u'batch_metrics_publish_interval': u'500', u'bias_init_sigma': u'0.01', u'_num_gpus': u'auto', u'_data_format': u'record', u'factors_wd': u'0.00001', u'linear_wd': u'0.001', u'_kvstore': u'auto', u'_learning_rate': u'1.0', u'_optimizer': u'adam'}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:09 INFO 139813082523456] Reading provided configuration from /opt/ml/input/config/hyperparameters.json: {u'epochs': u'200', u'feature_dim': u'2625', u'mini_batch_size': u'1000', u'predictor_type': u'binary_classifier', u'num_factors': u'64'}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:09 INFO 139813082523456] Final configuration: {u'factors_lr': u'0.0001', u'linear_init_sigma': u'0.01', u'epochs': u'200', u'feature_dim': u'2625', u'num_factors': u'64', u'_wd': u'1.0', u'_num_kv_servers': u'auto', u'use_bias': u'true', u'factors_init_sigma': u'0.001', u'_log_level': u'info', u'bias_init_method': u'normal', u'linear_init_method': u'normal', u'linear_lr': u'0.001', u'factors_init_method': u'normal', u'_tuning_objective_metric': u'', u'bias_wd': u'0.01', u'use_linear': u'true', u'bias_lr': u'0.1', u'mini_batch_size': u'1000', u'_use_full_symbolic': u'true', u'batch_metrics_publish_interval': u'500', u'predictor_type': u'binary_classifier', u'bias_init_sigma': u'0.01', u'_num_gpus': u'auto', u'_data_format': u'record', u'factors_wd': u'0.00001', u'linear_wd': u'0.001', u'_kvstore': u'auto', u'_learning_rate': u'1.0', u'_optimizer': u'adam'}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:09 WARNING 139813082523456] Loggers have already been setup.\u001b[0m\n",
      "\u001b[34mProcess 1 is a worker.\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:09 INFO 139813082523456] Using default worker.\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:09.397] [tensorio] [warning] TensorIO is already initialized; ignoring the initialization routine.\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:09.403] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 0, \"duration\": 11, \"num_examples\": 1, \"num_bytes\": 64000}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:09 INFO 139813082523456] nvidia-smi took: 0.0251219272614 secs to identify 0 gpus\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:09 INFO 139813082523456] Number of GPUs being used: 0\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:09 INFO 139813082523456] [Sparse network] Building a sparse network.\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:09 INFO 139813082523456] Create Store: local\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"initialize.time\": {\"count\": 1, \"max\": 35.70818901062012, \"sum\": 35.70818901062012, \"min\": 35.70818901062012}}, \"EndTime\": 1604871609.436315, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871609.391488}\n",
      "\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 1, \"sum\": 1.0, \"min\": 1}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 0, \"sum\": 0.0, \"min\": 0}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 0, \"sum\": 0.0, \"min\": 0}, \"Total Batches Seen\": {\"count\": 1, \"max\": 1, \"sum\": 1.0, \"min\": 1}, \"Total Records Seen\": {\"count\": 1, \"max\": 1000, \"sum\": 1000.0, \"min\": 1000}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 1000, \"sum\": 1000.0, \"min\": 1000}, \"Reset Count\": {\"count\": 1, \"max\": 1, \"sum\": 1.0, \"min\": 1}}, \"EndTime\": 1604871609.436486, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"init_train_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871609.436447}\n",
      "\u001b[0m\n",
      "\u001b[34m[21:40:09] /opt/brazil-pkg-cache/packages/AIAlgorithmsMXNet/AIAlgorithmsMXNet-1.1.x.203343.0/AL2012/generic-flavor/src/src/kvstore/./kvstore_local.h:280: Warning: non-default weights detected during kvstore pull. This call has been ignored. Please make sure to use row_sparse_pull with row_ids.\u001b[0m\n",
      "\u001b[34m[21:40:09] /opt/brazil-pkg-cache/packages/AIAlgorithmsMXNet/AIAlgorithmsMXNet-1.1.x.203343.0/AL2012/generic-flavor/src/src/kvstore/./kvstore_local.h:280: Warning: non-default weights detected during kvstore pull. This call has been ignored. Please make sure to use row_sparse_pull with row_ids.\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:09 INFO 139813082523456] #quality_metric: host=algo-1, epoch=0, batch=0 train binary_classification_accuracy <score>=0.526\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:09 INFO 139813082523456] #quality_metric: host=algo-1, epoch=0, batch=0 train binary_classification_cross_entropy <loss>=0.69270513916\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:09 INFO 139813082523456] #quality_metric: host=algo-1, epoch=0, batch=0 train binary_f_1.000 <score>=0.678861788618\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:10.046] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 2, \"duration\": 594, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:10 INFO 139813082523456] #quality_metric: host=algo-1, epoch=0, train binary_classification_accuracy <score>=0.548274725275\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:10 INFO 139813082523456] #quality_metric: host=algo-1, epoch=0, train binary_classification_cross_entropy <loss>=0.683914422674\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:10 INFO 139813082523456] #quality_metric: host=algo-1, epoch=0, train binary_f_1.000 <score>=0.708136009599\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"epochs\": {\"count\": 1, \"max\": 200, \"sum\": 200.0, \"min\": 200}, \"update.time\": {\"count\": 1, \"max\": 610.7211112976074, \"sum\": 610.7211112976074, \"min\": 610.7211112976074}}, \"EndTime\": 1604871610.047396, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871609.436385}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:10 INFO 139813082523456] #progress_metric: host=algo-1, completed 0 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 92, \"sum\": 92.0, \"min\": 92}, \"Total Records Seen\": {\"count\": 1, \"max\": 91570, \"sum\": 91570.0, \"min\": 91570}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 2, \"sum\": 2.0, \"min\": 2}}, \"EndTime\": 1604871610.047663, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 0}, \"StartTime\": 1604871609.436642}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:10 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=148200.873995 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:10 INFO 139813082523456] #quality_metric: host=algo-1, epoch=1, batch=0 train binary_classification_accuracy <score>=0.539\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:10 INFO 139813082523456] #quality_metric: host=algo-1, epoch=1, batch=0 train binary_classification_cross_entropy <loss>=0.679184692383\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:10 INFO 139813082523456] #quality_metric: host=algo-1, epoch=1, batch=0 train binary_f_1.000 <score>=0.700454840806\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:10.769] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 4, \"duration\": 719, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:10 INFO 139813082523456] #quality_metric: host=algo-1, epoch=1, train binary_classification_accuracy <score>=0.568450549451\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:10 INFO 139813082523456] #quality_metric: host=algo-1, epoch=1, train binary_classification_cross_entropy <loss>=0.67404157299\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:10 INFO 139813082523456] #quality_metric: host=algo-1, epoch=1, train binary_f_1.000 <score>=0.713599136516\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 721.2390899658203, \"sum\": 721.2390899658203, \"min\": 721.2390899658203}}, \"EndTime\": 1604871610.769575, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871610.047484}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:10 INFO 139813082523456] #progress_metric: host=algo-1, completed 1 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 183, \"sum\": 183.0, \"min\": 183}, \"Total Records Seen\": {\"count\": 1, \"max\": 182140, \"sum\": 182140.0, \"min\": 182140}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 3, \"sum\": 3.0, \"min\": 3}}, \"EndTime\": 1604871610.769802, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 1}, \"StartTime\": 1604871610.048304}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:10 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=125511.039522 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:10 INFO 139813082523456] #quality_metric: host=algo-1, epoch=2, batch=0 train binary_classification_accuracy <score>=0.586\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:10 INFO 139813082523456] #quality_metric: host=algo-1, epoch=2, batch=0 train binary_classification_cross_entropy <loss>=0.669577819824\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:10 INFO 139813082523456] #quality_metric: host=algo-1, epoch=2, batch=0 train binary_f_1.000 <score>=0.719891745602\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:11.506] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 6, \"duration\": 734, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:11 INFO 139813082523456] #quality_metric: host=algo-1, epoch=2, train binary_classification_accuracy <score>=0.601516483516\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:11 INFO 139813082523456] #quality_metric: host=algo-1, epoch=2, train binary_classification_cross_entropy <loss>=0.664631521372\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:11 INFO 139813082523456] #quality_metric: host=algo-1, epoch=2, train binary_f_1.000 <score>=0.726568037521\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 736.4871501922607, \"sum\": 736.4871501922607, \"min\": 736.4871501922607}}, \"EndTime\": 1604871611.50696, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871610.769645}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:11 INFO 139813082523456] #progress_metric: host=algo-1, completed 1 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 274, \"sum\": 274.0, \"min\": 274}, \"Total Records Seen\": {\"count\": 1, \"max\": 272710, \"sum\": 272710.0, \"min\": 272710}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 4, \"sum\": 4.0, \"min\": 4}}, \"EndTime\": 1604871611.507168, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 2}, \"StartTime\": 1604871610.77043}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:11 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=122915.783852 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:11 INFO 139813082523456] #quality_metric: host=algo-1, epoch=3, batch=0 train binary_classification_accuracy <score>=0.634\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:11 INFO 139813082523456] #quality_metric: host=algo-1, epoch=3, batch=0 train binary_classification_cross_entropy <loss>=0.660815429687\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:11 INFO 139813082523456] #quality_metric: host=algo-1, epoch=3, batch=0 train binary_f_1.000 <score>=0.73782234957\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:12.072] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 8, \"duration\": 563, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:12 INFO 139813082523456] #quality_metric: host=algo-1, epoch=3, train binary_classification_accuracy <score>=0.632945054945\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:12 INFO 139813082523456] #quality_metric: host=algo-1, epoch=3, train binary_classification_cross_entropy <loss>=0.655848484584\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:12 INFO 139813082523456] #quality_metric: host=algo-1, epoch=3, train binary_f_1.000 <score>=0.738642587753\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 565.467119216919, \"sum\": 565.467119216919, \"min\": 565.467119216919}}, \"EndTime\": 1604871612.073128, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871611.507027}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:12 INFO 139813082523456] #progress_metric: host=algo-1, completed 2 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 365, \"sum\": 365.0, \"min\": 365}, \"Total Records Seen\": {\"count\": 1, \"max\": 363280, \"sum\": 363280.0, \"min\": 363280}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 5, \"sum\": 5.0, \"min\": 5}}, \"EndTime\": 1604871612.073336, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 3}, \"StartTime\": 1604871611.507632}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:12 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=160074.784506 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:12 INFO 139813082523456] #quality_metric: host=algo-1, epoch=4, batch=0 train binary_classification_accuracy <score>=0.67\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:12 INFO 139813082523456] #quality_metric: host=algo-1, epoch=4, batch=0 train binary_classification_cross_entropy <loss>=0.652683166504\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:12 INFO 139813082523456] #quality_metric: host=algo-1, epoch=4, batch=0 train binary_f_1.000 <score>=0.7533632287\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:12.646] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 10, \"duration\": 571, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:12 INFO 139813082523456] #quality_metric: host=algo-1, epoch=4, train binary_classification_accuracy <score>=0.656934065934\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:12 INFO 139813082523456] #quality_metric: host=algo-1, epoch=4, train binary_classification_cross_entropy <loss>=0.647594163161\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:12 INFO 139813082523456] #quality_metric: host=algo-1, epoch=4, train binary_f_1.000 <score>=0.747367995145\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 573.5428333282471, \"sum\": 573.5428333282471, \"min\": 573.5428333282471}}, \"EndTime\": 1604871612.647392, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871612.073196}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:12 INFO 139813082523456] #progress_metric: host=algo-1, completed 2 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 456, \"sum\": 456.0, \"min\": 456}, \"Total Records Seen\": {\"count\": 1, \"max\": 453850, \"sum\": 453850.0, \"min\": 453850}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 6, \"sum\": 6.0, \"min\": 6}}, \"EndTime\": 1604871612.647584, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 4}, \"StartTime\": 1604871612.073819}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:12 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=157822.888039 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:12 INFO 139813082523456] #quality_metric: host=algo-1, epoch=5, batch=0 train binary_classification_accuracy <score>=0.694\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:12 INFO 139813082523456] #quality_metric: host=algo-1, epoch=5, batch=0 train binary_classification_cross_entropy <loss>=0.645190917969\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:12 INFO 139813082523456] #quality_metric: host=algo-1, epoch=5, batch=0 train binary_f_1.000 <score>=0.763888888889\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:13.215] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 12, \"duration\": 566, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:13 INFO 139813082523456] #quality_metric: host=algo-1, epoch=5, train binary_classification_accuracy <score>=0.673956043956\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:13 INFO 139813082523456] #quality_metric: host=algo-1, epoch=5, train binary_classification_cross_entropy <loss>=0.639911613045\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:13 INFO 139813082523456] #quality_metric: host=algo-1, epoch=5, train binary_f_1.000 <score>=0.753198356319\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 568.0339336395264, \"sum\": 568.0339336395264, \"min\": 568.0339336395264}}, \"EndTime\": 1604871613.216128, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871612.647456}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:13 INFO 139813082523456] #progress_metric: host=algo-1, completed 3 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 547, \"sum\": 547.0, \"min\": 547}, \"Total Records Seen\": {\"count\": 1, \"max\": 544420, \"sum\": 544420.0, \"min\": 544420}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 7, \"sum\": 7.0, \"min\": 7}}, \"EndTime\": 1604871613.216277, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 5}, \"StartTime\": 1604871612.648067}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:13 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=159370.316011 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:13 INFO 139813082523456] #quality_metric: host=algo-1, epoch=6, batch=0 train binary_classification_accuracy <score>=0.7\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:13 INFO 139813082523456] #quality_metric: host=algo-1, epoch=6, batch=0 train binary_classification_cross_entropy <loss>=0.638337646484\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:13 INFO 139813082523456] #quality_metric: host=algo-1, epoch=6, batch=0 train binary_f_1.000 <score>=0.764150943396\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:13.810] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 14, \"duration\": 593, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:13 INFO 139813082523456] #quality_metric: host=algo-1, epoch=6, train binary_classification_accuracy <score>=0.686032967033\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:13 INFO 139813082523456] #quality_metric: host=algo-1, epoch=6, train binary_classification_cross_entropy <loss>=0.632808052482\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:13 INFO 139813082523456] #quality_metric: host=algo-1, epoch=6, train binary_f_1.000 <score>=0.757158763482\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 594.7310924530029, \"sum\": 594.7310924530029, \"min\": 594.7310924530029}}, \"EndTime\": 1604871613.811502, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871613.216186}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:13 INFO 139813082523456] #progress_metric: host=algo-1, completed 3 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 638, \"sum\": 638.0, \"min\": 638}, \"Total Records Seen\": {\"count\": 1, \"max\": 634990, \"sum\": 634990.0, \"min\": 634990}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 8, \"sum\": 8.0, \"min\": 8}}, \"EndTime\": 1604871613.8117, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 6}, \"StartTime\": 1604871613.216741}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:13 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=152202.867092 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:13 INFO 139813082523456] #quality_metric: host=algo-1, epoch=7, batch=0 train binary_classification_accuracy <score>=0.713\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:13 INFO 139813082523456] #quality_metric: host=algo-1, epoch=7, batch=0 train binary_classification_cross_entropy <loss>=0.632094970703\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:13 INFO 139813082523456] #quality_metric: host=algo-1, epoch=7, batch=0 train binary_f_1.000 <score>=0.770216172938\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:14.400] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 16, \"duration\": 586, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:14 INFO 139813082523456] #quality_metric: host=algo-1, epoch=7, train binary_classification_accuracy <score>=0.695835164835\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:14 INFO 139813082523456] #quality_metric: host=algo-1, epoch=7, train binary_classification_cross_entropy <loss>=0.626263808701\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:14 INFO 139813082523456] #quality_metric: host=algo-1, epoch=7, train binary_f_1.000 <score>=0.760655454192\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 588.4919166564941, \"sum\": 588.4919166564941, \"min\": 588.4919166564941}}, \"EndTime\": 1604871614.4007, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871613.811568}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:14 INFO 139813082523456] #progress_metric: host=algo-1, completed 4 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 729, \"sum\": 729.0, \"min\": 729}, \"Total Records Seen\": {\"count\": 1, \"max\": 725560, \"sum\": 725560.0, \"min\": 725560}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 9, \"sum\": 9.0, \"min\": 9}}, \"EndTime\": 1604871614.400946, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 7}, \"StartTime\": 1604871613.812178}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:14 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=153799.359941 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:14 INFO 139813082523456] #quality_metric: host=algo-1, epoch=8, batch=0 train binary_classification_accuracy <score>=0.718\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:14 INFO 139813082523456] #quality_metric: host=algo-1, epoch=8, batch=0 train binary_classification_cross_entropy <loss>=0.626419921875\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:14 INFO 139813082523456] #quality_metric: host=algo-1, epoch=8, batch=0 train binary_f_1.000 <score>=0.769607843137\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:14.956] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 18, \"duration\": 554, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:14 INFO 139813082523456] #quality_metric: host=algo-1, epoch=8, train binary_classification_accuracy <score>=0.702120879121\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:14 INFO 139813082523456] #quality_metric: host=algo-1, epoch=8, train binary_classification_cross_entropy <loss>=0.620245735588\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:14 INFO 139813082523456] #quality_metric: host=algo-1, epoch=8, train binary_f_1.000 <score>=0.762250581064\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 555.6211471557617, \"sum\": 555.6211471557617, \"min\": 555.6211471557617}}, \"EndTime\": 1604871614.957099, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871614.400784}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:14 INFO 139813082523456] #progress_metric: host=algo-1, completed 4 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 820, \"sum\": 820.0, \"min\": 820}, \"Total Records Seen\": {\"count\": 1, \"max\": 816130, \"sum\": 816130.0, \"min\": 816130}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 10, \"sum\": 10.0, \"min\": 10}}, \"EndTime\": 1604871614.957342, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 8}, \"StartTime\": 1604871614.401449}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:14 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=162896.872612 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:14 INFO 139813082523456] #quality_metric: host=algo-1, epoch=9, batch=0 train binary_classification_accuracy <score>=0.714\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:14 INFO 139813082523456] #quality_metric: host=algo-1, epoch=9, batch=0 train binary_classification_cross_entropy <loss>=0.621264404297\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:14 INFO 139813082523456] #quality_metric: host=algo-1, epoch=9, batch=0 train binary_f_1.000 <score>=0.764415156507\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:15.526] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 20, \"duration\": 567, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:15 INFO 139813082523456] #quality_metric: host=algo-1, epoch=9, train binary_classification_accuracy <score>=0.707758241758\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:15 INFO 139813082523456] #quality_metric: host=algo-1, epoch=9, train binary_classification_cross_entropy <loss>=0.614714933626\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:15 INFO 139813082523456] #quality_metric: host=algo-1, epoch=9, train binary_f_1.000 <score>=0.763936230649\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 568.993091583252, \"sum\": 568.993091583252, \"min\": 568.993091583252}}, \"EndTime\": 1604871615.526851, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871614.957158}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:15 INFO 139813082523456] #progress_metric: host=algo-1, completed 5 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 911, \"sum\": 911.0, \"min\": 911}, \"Total Records Seen\": {\"count\": 1, \"max\": 906700, \"sum\": 906700.0, \"min\": 906700}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 11, \"sum\": 11.0, \"min\": 11}}, \"EndTime\": 1604871615.52705, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 9}, \"StartTime\": 1604871614.957829}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:15 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=159078.669659 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:15 INFO 139813082523456] #quality_metric: host=algo-1, epoch=10, batch=0 train binary_classification_accuracy <score>=0.715\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:15 INFO 139813082523456] #quality_metric: host=algo-1, epoch=10, batch=0 train binary_classification_cross_entropy <loss>=0.616579589844\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:15 INFO 139813082523456] #quality_metric: host=algo-1, epoch=10, batch=0 train binary_f_1.000 <score>=0.763877381939\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:16.085] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 22, \"duration\": 556, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:16 INFO 139813082523456] #quality_metric: host=algo-1, epoch=10, train binary_classification_accuracy <score>=0.712065934066\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:16 INFO 139813082523456] #quality_metric: host=algo-1, epoch=10, train binary_classification_cross_entropy <loss>=0.609630869436\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:16 INFO 139813082523456] #quality_metric: host=algo-1, epoch=10, train binary_f_1.000 <score>=0.765088757396\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 558.2261085510254, \"sum\": 558.2261085510254, \"min\": 558.2261085510254}}, \"EndTime\": 1604871616.085831, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871615.526912}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:16 INFO 139813082523456] #progress_metric: host=algo-1, completed 5 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 1002, \"sum\": 1002.0, \"min\": 1002}, \"Total Records Seen\": {\"count\": 1, \"max\": 997270, \"sum\": 997270.0, \"min\": 997270}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 12, \"sum\": 12.0, \"min\": 12}}, \"EndTime\": 1604871616.085985, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 10}, \"StartTime\": 1604871615.527579}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:16 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=162162.141295 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:16 INFO 139813082523456] #quality_metric: host=algo-1, epoch=11, batch=0 train binary_classification_accuracy <score>=0.715\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:16 INFO 139813082523456] #quality_metric: host=algo-1, epoch=11, batch=0 train binary_classification_cross_entropy <loss>=0.612318359375\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:16 INFO 139813082523456] #quality_metric: host=algo-1, epoch=11, batch=0 train binary_f_1.000 <score>=0.762301918265\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:16.667] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 24, \"duration\": 580, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:16 INFO 139813082523456] #quality_metric: host=algo-1, epoch=11, train binary_classification_accuracy <score>=0.715428571429\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:16 INFO 139813082523456] #quality_metric: host=algo-1, epoch=11, train binary_classification_cross_entropy <loss>=0.604953674987\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:16 INFO 139813082523456] #quality_metric: host=algo-1, epoch=11, train binary_f_1.000 <score>=0.765943600868\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 581.6621780395508, \"sum\": 581.6621780395508, \"min\": 581.6621780395508}}, \"EndTime\": 1604871616.668183, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871616.08589}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:16 INFO 139813082523456] #progress_metric: host=algo-1, completed 6 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 1093, \"sum\": 1093.0, \"min\": 1093}, \"Total Records Seen\": {\"count\": 1, \"max\": 1087840, \"sum\": 1087840.0, \"min\": 1087840}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 13, \"sum\": 13.0, \"min\": 13}}, \"EndTime\": 1604871616.66834, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 11}, \"StartTime\": 1604871616.086493}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:16 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=155634.618706 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:16 INFO 139813082523456] #quality_metric: host=algo-1, epoch=12, batch=0 train binary_classification_accuracy <score>=0.714\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:16 INFO 139813082523456] #quality_metric: host=algo-1, epoch=12, batch=0 train binary_classification_cross_entropy <loss>=0.608436889648\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:16 INFO 139813082523456] #quality_metric: host=algo-1, epoch=12, batch=0 train binary_f_1.000 <score>=0.760067114094\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:17.225] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 26, \"duration\": 555, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:17 INFO 139813082523456] #quality_metric: host=algo-1, epoch=12, train binary_classification_accuracy <score>=0.717571428571\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:17 INFO 139813082523456] #quality_metric: host=algo-1, epoch=12, train binary_classification_cross_entropy <loss>=0.600645390437\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:17 INFO 139813082523456] #quality_metric: host=algo-1, epoch=12, train binary_f_1.000 <score>=0.766224906539\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 556.9159984588623, \"sum\": 556.9159984588623, \"min\": 556.9159984588623}}, \"EndTime\": 1604871617.225764, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871616.668244}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:17 INFO 139813082523456] #progress_metric: host=algo-1, completed 6 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 1184, \"sum\": 1184.0, \"min\": 1184}, \"Total Records Seen\": {\"count\": 1, \"max\": 1178410, \"sum\": 1178410.0, \"min\": 1178410}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 14, \"sum\": 14.0, \"min\": 14}}, \"EndTime\": 1604871617.225943, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 12}, \"StartTime\": 1604871616.668819}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:17 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=162540.49874 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:17 INFO 139813082523456] #quality_metric: host=algo-1, epoch=13, batch=0 train binary_classification_accuracy <score>=0.713\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:17 INFO 139813082523456] #quality_metric: host=algo-1, epoch=13, batch=0 train binary_classification_cross_entropy <loss>=0.604894775391\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:17 INFO 139813082523456] #quality_metric: host=algo-1, epoch=13, batch=0 train binary_f_1.000 <score>=0.758620689655\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:17.796] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 28, \"duration\": 568, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:17 INFO 139813082523456] #quality_metric: host=algo-1, epoch=13, train binary_classification_accuracy <score>=0.719472527473\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:17 INFO 139813082523456] #quality_metric: host=algo-1, epoch=13, train binary_classification_cross_entropy <loss>=0.596670646248\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:17 INFO 139813082523456] #quality_metric: host=algo-1, epoch=13, train binary_f_1.000 <score>=0.766492261535\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 570.5060958862305, \"sum\": 570.5060958862305, \"min\": 570.5060958862305}}, \"EndTime\": 1604871617.796948, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871617.225822}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:17 INFO 139813082523456] #progress_metric: host=algo-1, completed 7 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 1275, \"sum\": 1275.0, \"min\": 1275}, \"Total Records Seen\": {\"count\": 1, \"max\": 1268980, \"sum\": 1268980.0, \"min\": 1268980}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 15, \"sum\": 15.0, \"min\": 15}}, \"EndTime\": 1604871617.797153, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 13}, \"StartTime\": 1604871617.226414}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:17 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=158661.48371 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:17 INFO 139813082523456] #quality_metric: host=algo-1, epoch=14, batch=0 train binary_classification_accuracy <score>=0.711\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:17 INFO 139813082523456] #quality_metric: host=algo-1, epoch=14, batch=0 train binary_classification_cross_entropy <loss>=0.601655334473\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:17 INFO 139813082523456] #quality_metric: host=algo-1, epoch=14, batch=0 train binary_f_1.000 <score>=0.75611814346\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "2020-11-08 21:40:07 Training - Training image download completed. Training in progress.\u001b[34m[2020-11-08 21:40:18.353] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 30, \"duration\": 554, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:18 INFO 139813082523456] #quality_metric: host=algo-1, epoch=14, train binary_classification_accuracy <score>=0.720879120879\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:18 INFO 139813082523456] #quality_metric: host=algo-1, epoch=14, train binary_classification_cross_entropy <loss>=0.592996871781\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:18 INFO 139813082523456] #quality_metric: host=algo-1, epoch=14, train binary_f_1.000 <score>=0.76650548804\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 556.4618110656738, \"sum\": 556.4618110656738, \"min\": 556.4618110656738}}, \"EndTime\": 1604871618.354108, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871617.797009}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:18 INFO 139813082523456] #progress_metric: host=algo-1, completed 7 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 1366, \"sum\": 1366.0, \"min\": 1366}, \"Total Records Seen\": {\"count\": 1, \"max\": 1359550, \"sum\": 1359550.0, \"min\": 1359550}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 16, \"sum\": 16.0, \"min\": 16}}, \"EndTime\": 1604871618.354328, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 14}, \"StartTime\": 1604871617.797618}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:18 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=162656.934355 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:18 INFO 139813082523456] #quality_metric: host=algo-1, epoch=15, batch=0 train binary_classification_accuracy <score>=0.711\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:18 INFO 139813082523456] #quality_metric: host=algo-1, epoch=15, batch=0 train binary_classification_cross_entropy <loss>=0.59868572998\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:18 INFO 139813082523456] #quality_metric: host=algo-1, epoch=15, batch=0 train binary_f_1.000 <score>=0.753623188406\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:18.961] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 32, \"duration\": 605, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:18 INFO 139813082523456] #quality_metric: host=algo-1, epoch=15, train binary_classification_accuracy <score>=0.722054945055\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:18 INFO 139813082523456] #quality_metric: host=algo-1, epoch=15, train binary_classification_cross_entropy <loss>=0.589594427423\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:18 INFO 139813082523456] #quality_metric: host=algo-1, epoch=15, train binary_f_1.000 <score>=0.766542675441\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 607.5968742370605, \"sum\": 607.5968742370605, \"min\": 607.5968742370605}}, \"EndTime\": 1604871618.962419, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871618.354175}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:18 INFO 139813082523456] #progress_metric: host=algo-1, completed 8 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 1457, \"sum\": 1457.0, \"min\": 1457}, \"Total Records Seen\": {\"count\": 1, \"max\": 1450120, \"sum\": 1450120.0, \"min\": 1450120}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 17, \"sum\": 17.0, \"min\": 17}}, \"EndTime\": 1604871618.962644, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 15}, \"StartTime\": 1604871618.354797}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:18 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=148975.84024 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:18 INFO 139813082523456] #quality_metric: host=algo-1, epoch=16, batch=0 train binary_classification_accuracy <score>=0.709\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:18 INFO 139813082523456] #quality_metric: host=algo-1, epoch=16, batch=0 train binary_classification_cross_entropy <loss>=0.595956542969\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:18 INFO 139813082523456] #quality_metric: host=algo-1, epoch=16, batch=0 train binary_f_1.000 <score>=0.751494449189\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:19.520] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 34, \"duration\": 556, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:19 INFO 139813082523456] #quality_metric: host=algo-1, epoch=16, train binary_classification_accuracy <score>=0.723318681319\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:19 INFO 139813082523456] #quality_metric: host=algo-1, epoch=16, train binary_classification_cross_entropy <loss>=0.586436416123\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:19 INFO 139813082523456] #quality_metric: host=algo-1, epoch=16, train binary_f_1.000 <score>=0.766801274452\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 558.107852935791, \"sum\": 558.107852935791, \"min\": 558.107852935791}}, \"EndTime\": 1604871619.521259, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871618.962474}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:19 INFO 139813082523456] #progress_metric: host=algo-1, completed 8 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 1548, \"sum\": 1548.0, \"min\": 1548}, \"Total Records Seen\": {\"count\": 1, \"max\": 1540690, \"sum\": 1540690.0, \"min\": 1540690}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 18, \"sum\": 18.0, \"min\": 18}}, \"EndTime\": 1604871619.521435, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 16}, \"StartTime\": 1604871618.963122}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:19 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=162192.05144 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:19 INFO 139813082523456] #quality_metric: host=algo-1, epoch=17, batch=0 train binary_classification_accuracy <score>=0.709\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:19 INFO 139813082523456] #quality_metric: host=algo-1, epoch=17, batch=0 train binary_classification_cross_entropy <loss>=0.59344140625\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:19 INFO 139813082523456] #quality_metric: host=algo-1, epoch=17, batch=0 train binary_f_1.000 <score>=0.750642673522\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:20.083] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 36, \"duration\": 560, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:20 INFO 139813082523456] #quality_metric: host=algo-1, epoch=17, train binary_classification_accuracy <score>=0.724450549451\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:20 INFO 139813082523456] #quality_metric: host=algo-1, epoch=17, train binary_classification_cross_entropy <loss>=0.583498621008\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:20 INFO 139813082523456] #quality_metric: host=algo-1, epoch=17, train binary_f_1.000 <score>=0.766950137088\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 562.1020793914795, \"sum\": 562.1020793914795, \"min\": 562.1020793914795}}, \"EndTime\": 1604871620.084027, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871619.521314}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:20 INFO 139813082523456] #progress_metric: host=algo-1, completed 9 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 1639, \"sum\": 1639.0, \"min\": 1639}, \"Total Records Seen\": {\"count\": 1, \"max\": 1631260, \"sum\": 1631260.0, \"min\": 1631260}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 19, \"sum\": 19.0, \"min\": 19}}, \"EndTime\": 1604871620.084182, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 17}, \"StartTime\": 1604871619.521899}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:20 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=161050.263075 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:20 INFO 139813082523456] #quality_metric: host=algo-1, epoch=18, batch=0 train binary_classification_accuracy <score>=0.708\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:20 INFO 139813082523456] #quality_metric: host=algo-1, epoch=18, batch=0 train binary_classification_cross_entropy <loss>=0.591116943359\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:20 INFO 139813082523456] #quality_metric: host=algo-1, epoch=18, batch=0 train binary_f_1.000 <score>=0.748709122203\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:20.664] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 38, \"duration\": 578, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:20 INFO 139813082523456] #quality_metric: host=algo-1, epoch=18, train binary_classification_accuracy <score>=0.725571428571\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:20 INFO 139813082523456] #quality_metric: host=algo-1, epoch=18, train binary_classification_cross_entropy <loss>=0.580759263929\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:20 INFO 139813082523456] #quality_metric: host=algo-1, epoch=18, train binary_f_1.000 <score>=0.767257849561\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 580.3098678588867, \"sum\": 580.3098678588867, \"min\": 580.3098678588867}}, \"EndTime\": 1604871620.664988, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871620.084084}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:20 INFO 139813082523456] #progress_metric: host=algo-1, completed 9 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 1730, \"sum\": 1730.0, \"min\": 1730}, \"Total Records Seen\": {\"count\": 1, \"max\": 1721830, \"sum\": 1721830.0, \"min\": 1721830}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 20, \"sum\": 20.0, \"min\": 20}}, \"EndTime\": 1604871620.665217, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 18}, \"StartTime\": 1604871620.084649}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:20 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=155971.504421 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:20 INFO 139813082523456] #quality_metric: host=algo-1, epoch=19, batch=0 train binary_classification_accuracy <score>=0.706\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:20 INFO 139813082523456] #quality_metric: host=algo-1, epoch=19, batch=0 train binary_classification_cross_entropy <loss>=0.588962524414\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:20 INFO 139813082523456] #quality_metric: host=algo-1, epoch=19, batch=0 train binary_f_1.000 <score>=0.746551724138\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:21.206] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 40, \"duration\": 539, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:21 INFO 139813082523456] #quality_metric: host=algo-1, epoch=19, train binary_classification_accuracy <score>=0.726582417582\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:21 INFO 139813082523456] #quality_metric: host=algo-1, epoch=19, train binary_classification_cross_entropy <loss>=0.578198819674\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:21 INFO 139813082523456] #quality_metric: host=algo-1, epoch=19, train binary_f_1.000 <score>=0.767460769928\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 541.5160655975342, \"sum\": 541.5160655975342, \"min\": 541.5160655975342}}, \"EndTime\": 1604871621.207255, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871620.665057}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:21 INFO 139813082523456] #progress_metric: host=algo-1, completed 10 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 1821, \"sum\": 1821.0, \"min\": 1821}, \"Total Records Seen\": {\"count\": 1, \"max\": 1812400, \"sum\": 1812400.0, \"min\": 1812400}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 21, \"sum\": 21.0, \"min\": 21}}, \"EndTime\": 1604871621.207414, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 19}, \"StartTime\": 1604871620.665713}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:21 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=167167.426252 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:21 INFO 139813082523456] #quality_metric: host=algo-1, epoch=20, batch=0 train binary_classification_accuracy <score>=0.708\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:21 INFO 139813082523456] #quality_metric: host=algo-1, epoch=20, batch=0 train binary_classification_cross_entropy <loss>=0.586959838867\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:21 INFO 139813082523456] #quality_metric: host=algo-1, epoch=20, batch=0 train binary_f_1.000 <score>=0.746967071057\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:21.786] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 42, \"duration\": 577, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:21 INFO 139813082523456] #quality_metric: host=algo-1, epoch=20, train binary_classification_accuracy <score>=0.727142857143\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:21 INFO 139813082523456] #quality_metric: host=algo-1, epoch=20, train binary_classification_cross_entropy <loss>=0.575799810724\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:21 INFO 139813082523456] #quality_metric: host=algo-1, epoch=20, train binary_f_1.000 <score>=0.767369959526\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 579.4119834899902, \"sum\": 579.4119834899902, \"min\": 579.4119834899902}}, \"EndTime\": 1604871621.787329, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871621.207316}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:21 INFO 139813082523456] #progress_metric: host=algo-1, completed 10 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 1912, \"sum\": 1912.0, \"min\": 1912}, \"Total Records Seen\": {\"count\": 1, \"max\": 1902970, \"sum\": 1902970.0, \"min\": 1902970}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 22, \"sum\": 22.0, \"min\": 22}}, \"EndTime\": 1604871621.787538, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 20}, \"StartTime\": 1604871621.207888}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:21 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=156219.281046 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:21 INFO 139813082523456] #quality_metric: host=algo-1, epoch=21, batch=0 train binary_classification_accuracy <score>=0.71\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:21 INFO 139813082523456] #quality_metric: host=algo-1, epoch=21, batch=0 train binary_classification_cross_entropy <loss>=0.585092651367\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:21 INFO 139813082523456] #quality_metric: host=algo-1, epoch=21, batch=0 train binary_f_1.000 <score>=0.747826086957\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:22.333] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 44, \"duration\": 544, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:22 INFO 139813082523456] #quality_metric: host=algo-1, epoch=21, train binary_classification_accuracy <score>=0.727868131868\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:22 INFO 139813082523456] #quality_metric: host=algo-1, epoch=21, train binary_classification_cross_entropy <loss>=0.573546600006\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:22 INFO 139813082523456] #quality_metric: host=algo-1, epoch=21, train binary_f_1.000 <score>=0.767452342943\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 545.8769798278809, \"sum\": 545.8769798278809, \"min\": 545.8769798278809}}, \"EndTime\": 1604871622.333968, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871621.787394}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:22 INFO 139813082523456] #progress_metric: host=algo-1, completed 11 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 2003, \"sum\": 2003.0, \"min\": 2003}, \"Total Records Seen\": {\"count\": 1, \"max\": 1993540, \"sum\": 1993540.0, \"min\": 1993540}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 23, \"sum\": 23.0, \"min\": 23}}, \"EndTime\": 1604871622.334142, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 21}, \"StartTime\": 1604871621.788063}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:22 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=165827.780298 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:22 INFO 139813082523456] #quality_metric: host=algo-1, epoch=22, batch=0 train binary_classification_accuracy <score>=0.712\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:22 INFO 139813082523456] #quality_metric: host=algo-1, epoch=22, batch=0 train binary_classification_cross_entropy <loss>=0.583346435547\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:22 INFO 139813082523456] #quality_metric: host=algo-1, epoch=22, batch=0 train binary_f_1.000 <score>=0.749128919861\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:22.890] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 46, \"duration\": 554, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:22 INFO 139813082523456] #quality_metric: host=algo-1, epoch=22, train binary_classification_accuracy <score>=0.728351648352\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:22 INFO 139813082523456] #quality_metric: host=algo-1, epoch=22, train binary_classification_cross_entropy <loss>=0.571425239982\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:22 INFO 139813082523456] #quality_metric: host=algo-1, epoch=22, train binary_f_1.000 <score>=0.767437484712\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 556.5390586853027, \"sum\": 556.5390586853027, \"min\": 556.5390586853027}}, \"EndTime\": 1604871622.891183, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871622.334042}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:22 INFO 139813082523456] #progress_metric: host=algo-1, completed 11 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 2094, \"sum\": 2094.0, \"min\": 2094}, \"Total Records Seen\": {\"count\": 1, \"max\": 2084110, \"sum\": 2084110.0, \"min\": 2084110}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 24, \"sum\": 24.0, \"min\": 24}}, \"EndTime\": 1604871622.891396, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 22}, \"StartTime\": 1604871622.334615}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:22 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=162639.454897 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:22 INFO 139813082523456] #quality_metric: host=algo-1, epoch=23, batch=0 train binary_classification_accuracy <score>=0.71\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:22 INFO 139813082523456] #quality_metric: host=algo-1, epoch=23, batch=0 train binary_classification_cross_entropy <loss>=0.581708679199\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:22 INFO 139813082523456] #quality_metric: host=algo-1, epoch=23, batch=0 train binary_f_1.000 <score>=0.746945898778\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:23.448] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 48, \"duration\": 555, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:23 INFO 139813082523456] #quality_metric: host=algo-1, epoch=23, train binary_classification_accuracy <score>=0.728846153846\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:23 INFO 139813082523456] #quality_metric: host=algo-1, epoch=23, train binary_classification_cross_entropy <loss>=0.569423246614\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:23 INFO 139813082523456] #quality_metric: host=algo-1, epoch=23, train binary_f_1.000 <score>=0.767495547787\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 557.1138858795166, \"sum\": 557.1138858795166, \"min\": 557.1138858795166}}, \"EndTime\": 1604871623.448995, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871622.891251}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:23 INFO 139813082523456] #progress_metric: host=algo-1, completed 12 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 2185, \"sum\": 2185.0, \"min\": 2185}, \"Total Records Seen\": {\"count\": 1, \"max\": 2174680, \"sum\": 2174680.0, \"min\": 2174680}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 25, \"sum\": 25.0, \"min\": 25}}, \"EndTime\": 1604871623.449218, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 23}, \"StartTime\": 1604871622.891851}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:23 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=162465.283621 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:23 INFO 139813082523456] #quality_metric: host=algo-1, epoch=24, batch=0 train binary_classification_accuracy <score>=0.71\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:23 INFO 139813082523456] #quality_metric: host=algo-1, epoch=24, batch=0 train binary_classification_cross_entropy <loss>=0.58016809082\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:23 INFO 139813082523456] #quality_metric: host=algo-1, epoch=24, batch=0 train binary_f_1.000 <score>=0.746503496503\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:24.037] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 50, \"duration\": 586, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:24 INFO 139813082523456] #quality_metric: host=algo-1, epoch=24, train binary_classification_accuracy <score>=0.729208791209\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:24 INFO 139813082523456] #quality_metric: host=algo-1, epoch=24, train binary_classification_cross_entropy <loss>=0.567529521565\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:24 INFO 139813082523456] #quality_metric: host=algo-1, epoch=24, train binary_f_1.000 <score>=0.767422983993\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 588.3259773254395, \"sum\": 588.3259773254395, \"min\": 588.3259773254395}}, \"EndTime\": 1604871624.03809, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871623.449063}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:24 INFO 139813082523456] #progress_metric: host=algo-1, completed 12 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 2276, \"sum\": 2276.0, \"min\": 2276}, \"Total Records Seen\": {\"count\": 1, \"max\": 2265250, \"sum\": 2265250.0, \"min\": 2265250}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 26, \"sum\": 26.0, \"min\": 26}}, \"EndTime\": 1604871624.038272, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 24}, \"StartTime\": 1604871623.449739}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:24 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=153867.324328 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:24 INFO 139813082523456] #quality_metric: host=algo-1, epoch=25, batch=0 train binary_classification_accuracy <score>=0.708\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:24 INFO 139813082523456] #quality_metric: host=algo-1, epoch=25, batch=0 train binary_classification_cross_entropy <loss>=0.57871472168\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:24 INFO 139813082523456] #quality_metric: host=algo-1, epoch=25, batch=0 train binary_f_1.000 <score>=0.744308231173\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:24.590] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 52, \"duration\": 550, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:24 INFO 139813082523456] #quality_metric: host=algo-1, epoch=25, train binary_classification_accuracy <score>=0.729659340659\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:24 INFO 139813082523456] #quality_metric: host=algo-1, epoch=25, train binary_classification_cross_entropy <loss>=0.565734155676\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:24 INFO 139813082523456] #quality_metric: host=algo-1, epoch=25, train binary_f_1.000 <score>=0.767434605458\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 552.2470474243164, \"sum\": 552.2470474243164, \"min\": 552.2470474243164}}, \"EndTime\": 1604871624.591003, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871624.038147}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:24 INFO 139813082523456] #progress_metric: host=algo-1, completed 13 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 2367, \"sum\": 2367.0, \"min\": 2367}, \"Total Records Seen\": {\"count\": 1, \"max\": 2355820, \"sum\": 2355820.0, \"min\": 2355820}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 27, \"sum\": 27.0, \"min\": 27}}, \"EndTime\": 1604871624.591158, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 25}, \"StartTime\": 1604871624.03873}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:24 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=163922.576654 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:24 INFO 139813082523456] #quality_metric: host=algo-1, epoch=26, batch=0 train binary_classification_accuracy <score>=0.71\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:24 INFO 139813082523456] #quality_metric: host=algo-1, epoch=26, batch=0 train binary_classification_cross_entropy <loss>=0.57733996582\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:24 INFO 139813082523456] #quality_metric: host=algo-1, epoch=26, batch=0 train binary_f_1.000 <score>=0.745614035088\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:25.150] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 54, \"duration\": 557, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:25 INFO 139813082523456] #quality_metric: host=algo-1, epoch=26, train binary_classification_accuracy <score>=0.730164835165\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:25 INFO 139813082523456] #quality_metric: host=algo-1, epoch=26, train binary_classification_cross_entropy <loss>=0.564028303545\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:25 INFO 139813082523456] #quality_metric: host=algo-1, epoch=26, train binary_f_1.000 <score>=0.767526627219\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 559.2021942138672, \"sum\": 559.2021942138672, \"min\": 559.2021942138672}}, \"EndTime\": 1604871625.150854, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871624.591062}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:25 INFO 139813082523456] #progress_metric: host=algo-1, completed 13 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 2458, \"sum\": 2458.0, \"min\": 2458}, \"Total Records Seen\": {\"count\": 1, \"max\": 2446390, \"sum\": 2446390.0, \"min\": 2446390}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 28, \"sum\": 28.0, \"min\": 28}}, \"EndTime\": 1604871625.151024, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 26}, \"StartTime\": 1604871624.591625}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:25 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=161879.50991 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:25 INFO 139813082523456] #quality_metric: host=algo-1, epoch=27, batch=0 train binary_classification_accuracy <score>=0.709\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:25 INFO 139813082523456] #quality_metric: host=algo-1, epoch=27, batch=0 train binary_classification_cross_entropy <loss>=0.576036193848\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:25 INFO 139813082523456] #quality_metric: host=algo-1, epoch=27, batch=0 train binary_f_1.000 <score>=0.744512730465\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:25.703] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 56, \"duration\": 550, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:25 INFO 139813082523456] #quality_metric: host=algo-1, epoch=27, train binary_classification_accuracy <score>=0.730714285714\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:25 INFO 139813082523456] #quality_metric: host=algo-1, epoch=27, train binary_classification_cross_entropy <loss>=0.562404110416\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:25 INFO 139813082523456] #quality_metric: host=algo-1, epoch=27, train binary_f_1.000 <score>=0.767722916805\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 552.232027053833, \"sum\": 552.232027053833, \"min\": 552.232027053833}}, \"EndTime\": 1604871625.703746, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871625.150916}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:25 INFO 139813082523456] #progress_metric: host=algo-1, completed 14 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 2549, \"sum\": 2549.0, \"min\": 2549}, \"Total Records Seen\": {\"count\": 1, \"max\": 2536960, \"sum\": 2536960.0, \"min\": 2536960}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 29, \"sum\": 29.0, \"min\": 29}}, \"EndTime\": 1604871625.703922, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 27}, \"StartTime\": 1604871625.151486}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:25 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=163918.686331 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:25 INFO 139813082523456] #quality_metric: host=algo-1, epoch=28, batch=0 train binary_classification_accuracy <score>=0.708\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:25 INFO 139813082523456] #quality_metric: host=algo-1, epoch=28, batch=0 train binary_classification_cross_entropy <loss>=0.574796569824\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:25 INFO 139813082523456] #quality_metric: host=algo-1, epoch=28, batch=0 train binary_f_1.000 <score>=0.743409490334\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:26.255] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 58, \"duration\": 550, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:26 INFO 139813082523456] #quality_metric: host=algo-1, epoch=28, train binary_classification_accuracy <score>=0.730967032967\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:26 INFO 139813082523456] #quality_metric: host=algo-1, epoch=28, train binary_classification_cross_entropy <loss>=0.560854569991\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:26 INFO 139813082523456] #quality_metric: host=algo-1, epoch=28, train binary_f_1.000 <score>=0.767696511937\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 552.0129203796387, \"sum\": 552.0129203796387, \"min\": 552.0129203796387}}, \"EndTime\": 1604871626.256443, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871625.703802}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:26 INFO 139813082523456] #progress_metric: host=algo-1, completed 14 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 2640, \"sum\": 2640.0, \"min\": 2640}, \"Total Records Seen\": {\"count\": 1, \"max\": 2627530, \"sum\": 2627530.0, \"min\": 2627530}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 30, \"sum\": 30.0, \"min\": 30}}, \"EndTime\": 1604871626.256633, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 28}, \"StartTime\": 1604871625.704402}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:26 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=163980.175032 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:26 INFO 139813082523456] #quality_metric: host=algo-1, epoch=29, batch=0 train binary_classification_accuracy <score>=0.709\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:26 INFO 139813082523456] #quality_metric: host=algo-1, epoch=29, batch=0 train binary_classification_cross_entropy <loss>=0.573615112305\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:26 INFO 139813082523456] #quality_metric: host=algo-1, epoch=29, batch=0 train binary_f_1.000 <score>=0.744063324538\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:26.835] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 60, \"duration\": 577, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:26 INFO 139813082523456] #quality_metric: host=algo-1, epoch=29, train binary_classification_accuracy <score>=0.731428571429\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:26 INFO 139813082523456] #quality_metric: host=algo-1, epoch=29, train binary_classification_cross_entropy <loss>=0.559373441256\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:26 INFO 139813082523456] #quality_metric: host=algo-1, epoch=29, train binary_f_1.000 <score>=0.767865962539\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 579.3099403381348, \"sum\": 579.3099403381348, \"min\": 579.3099403381348}}, \"EndTime\": 1604871626.836419, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871626.256499}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:26 INFO 139813082523456] #progress_metric: host=algo-1, completed 15 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 2731, \"sum\": 2731.0, \"min\": 2731}, \"Total Records Seen\": {\"count\": 1, \"max\": 2718100, \"sum\": 2718100.0, \"min\": 2718100}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 31, \"sum\": 31.0, \"min\": 31}}, \"EndTime\": 1604871626.836608, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 29}, \"StartTime\": 1604871626.257081}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:26 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=156254.301264 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:26 INFO 139813082523456] #quality_metric: host=algo-1, epoch=30, batch=0 train binary_classification_accuracy <score>=0.709\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:26 INFO 139813082523456] #quality_metric: host=algo-1, epoch=30, batch=0 train binary_classification_cross_entropy <loss>=0.572486694336\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:26 INFO 139813082523456] #quality_metric: host=algo-1, epoch=30, batch=0 train binary_f_1.000 <score>=0.744063324538\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:27.374] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 62, \"duration\": 535, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:27 INFO 139813082523456] #quality_metric: host=algo-1, epoch=30, train binary_classification_accuracy <score>=0.732021978022\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:27 INFO 139813082523456] #quality_metric: host=algo-1, epoch=30, train binary_classification_cross_entropy <loss>=0.557955168672\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:27 INFO 139813082523456] #quality_metric: host=algo-1, epoch=30, train binary_f_1.000 <score>=0.768158655309\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 537.3780727386475, \"sum\": 537.3780727386475, \"min\": 537.3780727386475}}, \"EndTime\": 1604871627.374498, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871626.836477}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:27 INFO 139813082523456] #progress_metric: host=algo-1, completed 15 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 2822, \"sum\": 2822.0, \"min\": 2822}, \"Total Records Seen\": {\"count\": 1, \"max\": 2808670, \"sum\": 2808670.0, \"min\": 2808670}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 32, \"sum\": 32.0, \"min\": 32}}, \"EndTime\": 1604871627.374656, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 30}, \"StartTime\": 1604871626.837094}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:27 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=168453.114558 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:27 INFO 139813082523456] #quality_metric: host=algo-1, epoch=31, batch=0 train binary_classification_accuracy <score>=0.708\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:27 INFO 139813082523456] #quality_metric: host=algo-1, epoch=31, batch=0 train binary_classification_cross_entropy <loss>=0.571406555176\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:27 INFO 139813082523456] #quality_metric: host=algo-1, epoch=31, batch=0 train binary_f_1.000 <score>=0.742957746479\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:27.932] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 64, \"duration\": 556, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:27 INFO 139813082523456] #quality_metric: host=algo-1, epoch=31, train binary_classification_accuracy <score>=0.732318681319\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:27 INFO 139813082523456] #quality_metric: host=algo-1, epoch=31, train binary_classification_cross_entropy <loss>=0.55659478961\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:27 INFO 139813082523456] #quality_metric: host=algo-1, epoch=31, train binary_f_1.000 <score>=0.76818834994\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 557.8570365905762, \"sum\": 557.8570365905762, \"min\": 557.8570365905762}}, \"EndTime\": 1604871627.933022, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871627.374559}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:27 INFO 139813082523456] #progress_metric: host=algo-1, completed 16 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 2913, \"sum\": 2913.0, \"min\": 2913}, \"Total Records Seen\": {\"count\": 1, \"max\": 2899240, \"sum\": 2899240.0, \"min\": 2899240}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 33, \"sum\": 33.0, \"min\": 33}}, \"EndTime\": 1604871627.933171, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 31}, \"StartTime\": 1604871627.375139}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:27 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=162276.024944 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:27 INFO 139813082523456] #quality_metric: host=algo-1, epoch=32, batch=0 train binary_classification_accuracy <score>=0.707\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:27 INFO 139813082523456] #quality_metric: host=algo-1, epoch=32, batch=0 train binary_classification_cross_entropy <loss>=0.570370788574\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:27 INFO 139813082523456] #quality_metric: host=algo-1, epoch=32, batch=0 train binary_f_1.000 <score>=0.741850220264\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[34m[2020-11-08 21:40:28.491] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 66, \"duration\": 556, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:28 INFO 139813082523456] #quality_metric: host=algo-1, epoch=32, train binary_classification_accuracy <score>=0.732791208791\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:28 INFO 139813082523456] #quality_metric: host=algo-1, epoch=32, train binary_classification_cross_entropy <loss>=0.555287888747\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:28 INFO 139813082523456] #quality_metric: host=algo-1, epoch=32, train binary_f_1.000 <score>=0.768383753715\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 558.3701133728027, \"sum\": 558.3701133728027, \"min\": 558.3701133728027}}, \"EndTime\": 1604871628.492029, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871627.933078}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:28 INFO 139813082523456] #progress_metric: host=algo-1, completed 16 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 3004, \"sum\": 3004.0, \"min\": 3004}, \"Total Records Seen\": {\"count\": 1, \"max\": 2989810, \"sum\": 2989810.0, \"min\": 2989810}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 34, \"sum\": 34.0, \"min\": 34}}, \"EndTime\": 1604871628.492203, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 32}, \"StartTime\": 1604871627.933634}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:28 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=162120.617724 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:28 INFO 139813082523456] #quality_metric: host=algo-1, epoch=33, batch=0 train binary_classification_accuracy <score>=0.704\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:28 INFO 139813082523456] #quality_metric: host=algo-1, epoch=33, batch=0 train binary_classification_cross_entropy <loss>=0.569375610352\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:28 INFO 139813082523456] #quality_metric: host=algo-1, epoch=33, batch=0 train binary_f_1.000 <score>=0.73897707231\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:29.070] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 68, \"duration\": 576, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:29 INFO 139813082523456] #quality_metric: host=algo-1, epoch=33, train binary_classification_accuracy <score>=0.732923076923\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:29 INFO 139813082523456] #quality_metric: host=algo-1, epoch=33, train binary_classification_cross_entropy <loss>=0.554030527639\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:29 INFO 139813082523456] #quality_metric: host=algo-1, epoch=33, train binary_f_1.000 <score>=0.768321513002\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 577.876091003418, \"sum\": 577.876091003418, \"min\": 577.876091003418}}, \"EndTime\": 1604871629.07058, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871628.492097}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:29 INFO 139813082523456] #progress_metric: host=algo-1, completed 17 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 3095, \"sum\": 3095.0, \"min\": 3095}, \"Total Records Seen\": {\"count\": 1, \"max\": 3080380, \"sum\": 3080380.0, \"min\": 3080380}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 35, \"sum\": 35.0, \"min\": 35}}, \"EndTime\": 1604871629.070761, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 33}, \"StartTime\": 1604871628.492676}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:29 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=156647.280135 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:29 INFO 139813082523456] #quality_metric: host=algo-1, epoch=34, batch=0 train binary_classification_accuracy <score>=0.707\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:29 INFO 139813082523456] #quality_metric: host=algo-1, epoch=34, batch=0 train binary_classification_cross_entropy <loss>=0.56841809082\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:29 INFO 139813082523456] #quality_metric: host=algo-1, epoch=34, batch=0 train binary_f_1.000 <score>=0.740937223696\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:29.639] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 70, \"duration\": 566, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:29 INFO 139813082523456] #quality_metric: host=algo-1, epoch=34, train binary_classification_accuracy <score>=0.733340659341\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:29 INFO 139813082523456] #quality_metric: host=algo-1, epoch=34, train binary_classification_cross_entropy <loss>=0.552819181673\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:29 INFO 139813082523456] #quality_metric: host=algo-1, epoch=34, train binary_f_1.000 <score>=0.76844536051\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 568.580150604248, \"sum\": 568.580150604248, \"min\": 568.580150604248}}, \"EndTime\": 1604871629.639833, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871629.070636}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:29 INFO 139813082523456] #progress_metric: host=algo-1, completed 17 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 3186, \"sum\": 3186.0, \"min\": 3186}, \"Total Records Seen\": {\"count\": 1, \"max\": 3170950, \"sum\": 3170950.0, \"min\": 3170950}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 36, \"sum\": 36.0, \"min\": 36}}, \"EndTime\": 1604871629.640023, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 34}, \"StartTime\": 1604871629.071223}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:29 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=159203.673119 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:29 INFO 139813082523456] #quality_metric: host=algo-1, epoch=35, batch=0 train binary_classification_accuracy <score>=0.708\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:29 INFO 139813082523456] #quality_metric: host=algo-1, epoch=35, batch=0 train binary_classification_cross_entropy <loss>=0.567495239258\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:29 INFO 139813082523456] #quality_metric: host=algo-1, epoch=35, batch=0 train binary_f_1.000 <score>=0.741592920354\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:30.207] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 72, \"duration\": 566, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:30 INFO 139813082523456] #quality_metric: host=algo-1, epoch=35, train binary_classification_accuracy <score>=0.733604395604\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:30 INFO 139813082523456] #quality_metric: host=algo-1, epoch=35, train binary_classification_cross_entropy <loss>=0.551650703849\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:30 INFO 139813082523456] #quality_metric: host=algo-1, epoch=35, train binary_f_1.000 <score>=0.768488807396\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 567.4810409545898, \"sum\": 567.4810409545898, \"min\": 567.4810409545898}}, \"EndTime\": 1604871630.207991, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871629.639893}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:30 INFO 139813082523456] #progress_metric: host=algo-1, completed 18 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 3277, \"sum\": 3277.0, \"min\": 3277}, \"Total Records Seen\": {\"count\": 1, \"max\": 3261520, \"sum\": 3261520.0, \"min\": 3261520}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 37, \"sum\": 37.0, \"min\": 37}}, \"EndTime\": 1604871630.208201, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 35}, \"StartTime\": 1604871629.64048}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:30 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=159492.698046 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:30 INFO 139813082523456] #quality_metric: host=algo-1, epoch=36, batch=0 train binary_classification_accuracy <score>=0.707\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:30 INFO 139813082523456] #quality_metric: host=algo-1, epoch=36, batch=0 train binary_classification_cross_entropy <loss>=0.566604736328\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:30 INFO 139813082523456] #quality_metric: host=algo-1, epoch=36, batch=0 train binary_f_1.000 <score>=0.74047829938\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:30.771] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 74, \"duration\": 561, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:30 INFO 139813082523456] #quality_metric: host=algo-1, epoch=36, train binary_classification_accuracy <score>=0.733956043956\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:30 INFO 139813082523456] #quality_metric: host=algo-1, epoch=36, train binary_classification_cross_entropy <loss>=0.550522261735\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:30 INFO 139813082523456] #quality_metric: host=algo-1, epoch=36, train binary_f_1.000 <score>=0.768661850705\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 563.3180141448975, \"sum\": 563.3180141448975, \"min\": 563.3180141448975}}, \"EndTime\": 1604871630.772173, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871630.208053}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:30 INFO 139813082523456] #progress_metric: host=algo-1, completed 18 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 3368, \"sum\": 3368.0, \"min\": 3368}, \"Total Records Seen\": {\"count\": 1, \"max\": 3352090, \"sum\": 3352090.0, \"min\": 3352090}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 38, \"sum\": 38.0, \"min\": 38}}, \"EndTime\": 1604871630.772326, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 36}, \"StartTime\": 1604871630.208829}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:30 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=160702.526369 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:30 INFO 139813082523456] #quality_metric: host=algo-1, epoch=37, batch=0 train binary_classification_accuracy <score>=0.707\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:30 INFO 139813082523456] #quality_metric: host=algo-1, epoch=37, batch=0 train binary_classification_cross_entropy <loss>=0.565744384766\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:30 INFO 139813082523456] #quality_metric: host=algo-1, epoch=37, batch=0 train binary_f_1.000 <score>=0.740017746229\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:31.318] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 76, \"duration\": 544, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:31 INFO 139813082523456] #quality_metric: host=algo-1, epoch=37, train binary_classification_accuracy <score>=0.734076923077\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:31 INFO 139813082523456] #quality_metric: host=algo-1, epoch=37, train binary_classification_cross_entropy <loss>=0.549431332096\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:31 INFO 139813082523456] #quality_metric: host=algo-1, epoch=37, train binary_f_1.000 <score>=0.768658642678\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 546.6141700744629, \"sum\": 546.6141700744629, \"min\": 546.6141700744629}}, \"EndTime\": 1604871631.319439, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871630.772231}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:31 INFO 139813082523456] #progress_metric: host=algo-1, completed 19 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 3459, \"sum\": 3459.0, \"min\": 3459}, \"Total Records Seen\": {\"count\": 1, \"max\": 3442660, \"sum\": 3442660.0, \"min\": 3442660}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 39, \"sum\": 39.0, \"min\": 39}}, \"EndTime\": 1604871631.319642, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 37}, \"StartTime\": 1604871630.772794}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:31 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=165590.973826 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:31 INFO 139813082523456] #quality_metric: host=algo-1, epoch=38, batch=0 train binary_classification_accuracy <score>=0.708\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:31 INFO 139813082523456] #quality_metric: host=algo-1, epoch=38, batch=0 train binary_classification_cross_entropy <loss>=0.564912109375\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:31 INFO 139813082523456] #quality_metric: host=algo-1, epoch=38, batch=0 train binary_f_1.000 <score>=0.740674955595\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:31.891] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 78, \"duration\": 569, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:31 INFO 139813082523456] #quality_metric: host=algo-1, epoch=38, train binary_classification_accuracy <score>=0.734428571429\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:31 INFO 139813082523456] #quality_metric: host=algo-1, epoch=38, train binary_classification_cross_entropy <loss>=0.548375619742\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:31 INFO 139813082523456] #quality_metric: host=algo-1, epoch=38, train binary_f_1.000 <score>=0.768831963881\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 571.3620185852051, \"sum\": 571.3620185852051, \"min\": 571.3620185852051}}, \"EndTime\": 1604871631.891481, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871631.319497}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:31 INFO 139813082523456] #progress_metric: host=algo-1, completed 19 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 3550, \"sum\": 3550.0, \"min\": 3550}, \"Total Records Seen\": {\"count\": 1, \"max\": 3533230, \"sum\": 3533230.0, \"min\": 3533230}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 40, \"sum\": 40.0, \"min\": 40}}, \"EndTime\": 1604871631.89162, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 38}, \"StartTime\": 1604871631.320095}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:31 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=158445.812406 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:31 INFO 139813082523456] #quality_metric: host=algo-1, epoch=39, batch=0 train binary_classification_accuracy <score>=0.71\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:31 INFO 139813082523456] #quality_metric: host=algo-1, epoch=39, batch=0 train binary_classification_cross_entropy <loss>=0.564106201172\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:31 INFO 139813082523456] #quality_metric: host=algo-1, epoch=39, batch=0 train binary_f_1.000 <score>=0.741992882562\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:32.428] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 80, \"duration\": 535, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:32 INFO 139813082523456] #quality_metric: host=algo-1, epoch=39, train binary_classification_accuracy <score>=0.734637362637\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:32 INFO 139813082523456] #quality_metric: host=algo-1, epoch=39, train binary_classification_cross_entropy <loss>=0.547353074965\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:32 INFO 139813082523456] #quality_metric: host=algo-1, epoch=39, train binary_f_1.000 <score>=0.768883273994\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 537.1568202972412, \"sum\": 537.1568202972412, \"min\": 537.1568202972412}}, \"EndTime\": 1604871632.429228, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871631.89153}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:32 INFO 139813082523456] #progress_metric: host=algo-1, completed 20 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 3641, \"sum\": 3641.0, \"min\": 3641}, \"Total Records Seen\": {\"count\": 1, \"max\": 3623800, \"sum\": 3623800.0, \"min\": 3623800}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 41, \"sum\": 41.0, \"min\": 41}}, \"EndTime\": 1604871632.429423, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 39}, \"StartTime\": 1604871631.892043}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:32 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=168507.288628 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:32 INFO 139813082523456] #quality_metric: host=algo-1, epoch=40, batch=0 train binary_classification_accuracy <score>=0.71\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:32 INFO 139813082523456] #quality_metric: host=algo-1, epoch=40, batch=0 train binary_classification_cross_entropy <loss>=0.563325073242\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:32 INFO 139813082523456] #quality_metric: host=algo-1, epoch=40, batch=0 train binary_f_1.000 <score>=0.741992882562\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:32.998] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 82, \"duration\": 567, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:32 INFO 139813082523456] #quality_metric: host=algo-1, epoch=40, train binary_classification_accuracy <score>=0.734835164835\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:32 INFO 139813082523456] #quality_metric: host=algo-1, epoch=40, train binary_classification_cross_entropy <loss>=0.546361840552\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:32 INFO 139813082523456] #quality_metric: host=algo-1, epoch=40, train binary_f_1.000 <score>=0.768953829066\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 569.2808628082275, \"sum\": 569.2808628082275, \"min\": 569.2808628082275}}, \"EndTime\": 1604871632.999233, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871632.429288}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:32 INFO 139813082523456] #progress_metric: host=algo-1, completed 20 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 3732, \"sum\": 3732.0, \"min\": 3732}, \"Total Records Seen\": {\"count\": 1, \"max\": 3714370, \"sum\": 3714370.0, \"min\": 3714370}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 42, \"sum\": 42.0, \"min\": 42}}, \"EndTime\": 1604871632.999443, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 40}, \"StartTime\": 1604871632.429923}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:32 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=158999.569006 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:33 INFO 139813082523456] #quality_metric: host=algo-1, epoch=41, batch=0 train binary_classification_accuracy <score>=0.709\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:33 INFO 139813082523456] #quality_metric: host=algo-1, epoch=41, batch=0 train binary_classification_cross_entropy <loss>=0.562567382812\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:33 INFO 139813082523456] #quality_metric: host=algo-1, epoch=41, batch=0 train binary_f_1.000 <score>=0.740872662511\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:33.572] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 84, \"duration\": 570, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:33 INFO 139813082523456] #quality_metric: host=algo-1, epoch=41, train binary_classification_accuracy <score>=0.735252747253\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:33 INFO 139813082523456] #quality_metric: host=algo-1, epoch=41, train binary_classification_cross_entropy <loss>=0.545400222946\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:33 INFO 139813082523456] #quality_metric: host=algo-1, epoch=41, train binary_f_1.000 <score>=0.769246978143\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 572.9858875274658, \"sum\": 572.9858875274658, \"min\": 572.9858875274658}}, \"EndTime\": 1604871633.57296, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871632.999292}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:33 INFO 139813082523456] #progress_metric: host=algo-1, completed 21 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 3823, \"sum\": 3823.0, \"min\": 3823}, \"Total Records Seen\": {\"count\": 1, \"max\": 3804940, \"sum\": 3804940.0, \"min\": 3804940}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 43, \"sum\": 43.0, \"min\": 43}}, \"EndTime\": 1604871633.573184, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 41}, \"StartTime\": 1604871632.999946}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:33 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=157965.497283 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:33 INFO 139813082523456] #quality_metric: host=algo-1, epoch=42, batch=0 train binary_classification_accuracy <score>=0.709\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:33 INFO 139813082523456] #quality_metric: host=algo-1, epoch=42, batch=0 train binary_classification_cross_entropy <loss>=0.56183190918\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:33 INFO 139813082523456] #quality_metric: host=algo-1, epoch=42, batch=0 train binary_f_1.000 <score>=0.740872662511\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:34.191] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 86, \"duration\": 616, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:34 INFO 139813082523456] #quality_metric: host=algo-1, epoch=42, train binary_classification_accuracy <score>=0.735593406593\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:34 INFO 139813082523456] #quality_metric: host=algo-1, epoch=42, train binary_classification_cross_entropy <loss>=0.544466697609\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:34 INFO 139813082523456] #quality_metric: host=algo-1, epoch=42, train binary_f_1.000 <score>=0.769484283237\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 618.3040142059326, \"sum\": 618.3040142059326, \"min\": 618.3040142059326}}, \"EndTime\": 1604871634.192025, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871633.573029}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:34 INFO 139813082523456] #progress_metric: host=algo-1, completed 21 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 3914, \"sum\": 3914.0, \"min\": 3914}, \"Total Records Seen\": {\"count\": 1, \"max\": 3895510, \"sum\": 3895510.0, \"min\": 3895510}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 44, \"sum\": 44.0, \"min\": 44}}, \"EndTime\": 1604871634.19225, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 42}, \"StartTime\": 1604871633.57369}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:34 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=146398.017166 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:34 INFO 139813082523456] #quality_metric: host=algo-1, epoch=43, batch=0 train binary_classification_accuracy <score>=0.71\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:34 INFO 139813082523456] #quality_metric: host=algo-1, epoch=43, batch=0 train binary_classification_cross_entropy <loss>=0.561117431641\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:34 INFO 139813082523456] #quality_metric: host=algo-1, epoch=43, batch=0 train binary_f_1.000 <score>=0.741532976827\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:34.781] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 88, \"duration\": 587, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:34 INFO 139813082523456] #quality_metric: host=algo-1, epoch=43, train binary_classification_accuracy <score>=0.735956043956\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:34 INFO 139813082523456] #quality_metric: host=algo-1, epoch=43, train binary_classification_cross_entropy <loss>=0.543559845977\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:34 INFO 139813082523456] #quality_metric: host=algo-1, epoch=43, train binary_f_1.000 <score>=0.769758528172\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 588.921070098877, \"sum\": 588.921070098877, \"min\": 588.921070098877}}, \"EndTime\": 1604871634.781677, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871634.19211}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:34 INFO 139813082523456] #progress_metric: host=algo-1, completed 22 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 4005, \"sum\": 4005.0, \"min\": 4005}, \"Total Records Seen\": {\"count\": 1, \"max\": 3986080, \"sum\": 3986080.0, \"min\": 3986080}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 45, \"sum\": 45.0, \"min\": 45}}, \"EndTime\": 1604871634.781866, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 43}, \"StartTime\": 1604871634.192727}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:34 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=153708.564589 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:34 INFO 139813082523456] #quality_metric: host=algo-1, epoch=44, batch=0 train binary_classification_accuracy <score>=0.71\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:34 INFO 139813082523456] #quality_metric: host=algo-1, epoch=44, batch=0 train binary_classification_cross_entropy <loss>=0.560422973633\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:34 INFO 139813082523456] #quality_metric: host=algo-1, epoch=44, batch=0 train binary_f_1.000 <score>=0.741532976827\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:35.353] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 90, \"duration\": 570, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:35 INFO 139813082523456] #quality_metric: host=algo-1, epoch=44, train binary_classification_accuracy <score>=0.736065934066\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:35 INFO 139813082523456] #quality_metric: host=algo-1, epoch=44, train binary_classification_cross_entropy <loss>=0.542678415822\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:35 INFO 139813082523456] #quality_metric: host=algo-1, epoch=44, train binary_f_1.000 <score>=0.769770518203\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 571.8591213226318, \"sum\": 571.8591213226318, \"min\": 571.8591213226318}}, \"EndTime\": 1604871635.354246, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871634.781734}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:35 INFO 139813082523456] #progress_metric: host=algo-1, completed 22 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 4096, \"sum\": 4096.0, \"min\": 4096}, \"Total Records Seen\": {\"count\": 1, \"max\": 4076650, \"sum\": 4076650.0, \"min\": 4076650}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 46, \"sum\": 46.0, \"min\": 46}}, \"EndTime\": 1604871635.354426, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 44}, \"StartTime\": 1604871634.78236}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:35 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=158294.28736 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:35 INFO 139813082523456] #quality_metric: host=algo-1, epoch=45, batch=0 train binary_classification_accuracy <score>=0.711\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:35 INFO 139813082523456] #quality_metric: host=algo-1, epoch=45, batch=0 train binary_classification_cross_entropy <loss>=0.559747436523\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:35 INFO 139813082523456] #quality_metric: host=algo-1, epoch=45, batch=0 train binary_f_1.000 <score>=0.742194469224\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:35.925] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 92, \"duration\": 569, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:35 INFO 139813082523456] #quality_metric: host=algo-1, epoch=45, train binary_classification_accuracy <score>=0.741516483516\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:35 INFO 139813082523456] #quality_metric: host=algo-1, epoch=45, train binary_classification_cross_entropy <loss>=0.541821194492\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:35 INFO 139813082523456] #quality_metric: host=algo-1, epoch=45, train binary_f_1.000 <score>=0.773439155478\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 571.1531639099121, \"sum\": 571.1531639099121, \"min\": 571.1531639099121}}, \"EndTime\": 1604871635.926102, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871635.354299}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:35 INFO 139813082523456] #progress_metric: host=algo-1, completed 23 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 4187, \"sum\": 4187.0, \"min\": 4187}, \"Total Records Seen\": {\"count\": 1, \"max\": 4167220, \"sum\": 4167220.0, \"min\": 4167220}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 47, \"sum\": 47.0, \"min\": 47}}, \"EndTime\": 1604871635.926333, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 45}, \"StartTime\": 1604871635.354917}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:35 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=158465.707147 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:35 INFO 139813082523456] #quality_metric: host=algo-1, epoch=46, batch=0 train binary_classification_accuracy <score>=0.713\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:35 INFO 139813082523456] #quality_metric: host=algo-1, epoch=46, batch=0 train binary_classification_cross_entropy <loss>=0.559090148926\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:35 INFO 139813082523456] #quality_metric: host=algo-1, epoch=46, batch=0 train binary_f_1.000 <score>=0.743521000894\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:36.496] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 94, \"duration\": 568, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:36 INFO 139813082523456] #quality_metric: host=algo-1, epoch=46, train binary_classification_accuracy <score>=0.74156043956\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:36 INFO 139813082523456] #quality_metric: host=algo-1, epoch=46, train binary_classification_cross_entropy <loss>=0.540987121582\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:36 INFO 139813082523456] #quality_metric: host=algo-1, epoch=46, train binary_f_1.000 <score>=0.773420941077\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 570.2989101409912, \"sum\": 570.2989101409912, \"min\": 570.2989101409912}}, \"EndTime\": 1604871636.497232, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871635.926169}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:36 INFO 139813082523456] #progress_metric: host=algo-1, completed 23 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 4278, \"sum\": 4278.0, \"min\": 4278}, \"Total Records Seen\": {\"count\": 1, \"max\": 4257790, \"sum\": 4257790.0, \"min\": 4257790}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 48, \"sum\": 48.0, \"min\": 48}}, \"EndTime\": 1604871636.497444, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 46}, \"StartTime\": 1604871635.926906}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:36 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=158718.493997 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:36 INFO 139813082523456] #quality_metric: host=algo-1, epoch=47, batch=0 train binary_classification_accuracy <score>=0.714\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:36 INFO 139813082523456] #quality_metric: host=algo-1, epoch=47, batch=0 train binary_classification_cross_entropy <loss>=0.558450195312\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:36 INFO 139813082523456] #quality_metric: host=algo-1, epoch=47, batch=0 train binary_f_1.000 <score>=0.744186046512\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:37.107] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 96, \"duration\": 608, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:37 INFO 139813082523456] #quality_metric: host=algo-1, epoch=47, train binary_classification_accuracy <score>=0.741813186813\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:37 INFO 139813082523456] #quality_metric: host=algo-1, epoch=47, train binary_classification_cross_entropy <loss>=0.540175176935\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:37 INFO 139813082523456] #quality_metric: host=algo-1, epoch=47, train binary_f_1.000 <score>=0.773574904833\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 610.0010871887207, \"sum\": 610.0010871887207, \"min\": 610.0010871887207}}, \"EndTime\": 1604871637.107941, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871636.497297}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:37 INFO 139813082523456] #progress_metric: host=algo-1, completed 24 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 4369, \"sum\": 4369.0, \"min\": 4369}, \"Total Records Seen\": {\"count\": 1, \"max\": 4348360, \"sum\": 4348360.0, \"min\": 4348360}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 49, \"sum\": 49.0, \"min\": 49}}, \"EndTime\": 1604871637.108215, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 47}, \"StartTime\": 1604871636.49791}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:37 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=148369.603093 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:37 INFO 139813082523456] #quality_metric: host=algo-1, epoch=48, batch=0 train binary_classification_accuracy <score>=0.712\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:37 INFO 139813082523456] #quality_metric: host=algo-1, epoch=48, batch=0 train binary_classification_cross_entropy <loss>=0.557826782227\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:37 INFO 139813082523456] #quality_metric: host=algo-1, epoch=48, batch=0 train binary_f_1.000 <score>=0.741935483871\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:37.683] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 98, \"duration\": 573, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:37 INFO 139813082523456] #quality_metric: host=algo-1, epoch=48, train binary_classification_accuracy <score>=0.742208791209\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:37 INFO 139813082523456] #quality_metric: host=algo-1, epoch=48, train binary_classification_cross_entropy <loss>=0.539384421548\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:37 INFO 139813082523456] #quality_metric: host=algo-1, epoch=48, train binary_f_1.000 <score>=0.773821575603\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 574.7919082641602, \"sum\": 574.7919082641602, \"min\": 574.7919082641602}}, \"EndTime\": 1604871637.683747, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871637.108023}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:37 INFO 139813082523456] #progress_metric: host=algo-1, completed 24 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 4460, \"sum\": 4460.0, \"min\": 4460}, \"Total Records Seen\": {\"count\": 1, \"max\": 4438930, \"sum\": 4438930.0, \"min\": 4438930}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 50, \"sum\": 50.0, \"min\": 50}}, \"EndTime\": 1604871637.683897, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 48}, \"StartTime\": 1604871637.108927}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:37 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=157495.790105 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:37 INFO 139813082523456] #quality_metric: host=algo-1, epoch=49, batch=0 train binary_classification_accuracy <score>=0.71\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:37 INFO 139813082523456] #quality_metric: host=algo-1, epoch=49, batch=0 train binary_classification_cross_entropy <loss>=0.557219299316\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:37 INFO 139813082523456] #quality_metric: host=algo-1, epoch=49, batch=0 train binary_f_1.000 <score>=0.739676840215\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[34m[2020-11-08 21:40:38.234] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 100, \"duration\": 548, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:38 INFO 139813082523456] #quality_metric: host=algo-1, epoch=49, train binary_classification_accuracy <score>=0.742164835165\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:38 INFO 139813082523456] #quality_metric: host=algo-1, epoch=49, train binary_classification_cross_entropy <loss>=0.538613998245\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:38 INFO 139813082523456] #quality_metric: host=algo-1, epoch=49, train binary_f_1.000 <score>=0.773673904446\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 550.3809452056885, \"sum\": 550.3809452056885, \"min\": 550.3809452056885}}, \"EndTime\": 1604871638.23479, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871637.683804}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:38 INFO 139813082523456] #progress_metric: host=algo-1, completed 25 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 4551, \"sum\": 4551.0, \"min\": 4551}, \"Total Records Seen\": {\"count\": 1, \"max\": 4529500, \"sum\": 4529500.0, \"min\": 4529500}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 51, \"sum\": 51.0, \"min\": 51}}, \"EndTime\": 1604871638.234958, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 49}, \"StartTime\": 1604871637.684383}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:38 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=164475.103373 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:38 INFO 139813082523456] #quality_metric: host=algo-1, epoch=50, batch=0 train binary_classification_accuracy <score>=0.71\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:38 INFO 139813082523456] #quality_metric: host=algo-1, epoch=50, batch=0 train binary_classification_cross_entropy <loss>=0.556627075195\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:38 INFO 139813082523456] #quality_metric: host=algo-1, epoch=50, batch=0 train binary_f_1.000 <score>=0.739676840215\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:38.780] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 102, \"duration\": 543, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:38 INFO 139813082523456] #quality_metric: host=algo-1, epoch=50, train binary_classification_accuracy <score>=0.742461538462\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:38 INFO 139813082523456] #quality_metric: host=algo-1, epoch=50, train binary_classification_cross_entropy <loss>=0.537863073328\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:38 INFO 139813082523456] #quality_metric: host=algo-1, epoch=50, train binary_f_1.000 <score>=0.773853635943\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 545.1259613037109, \"sum\": 545.1259613037109, \"min\": 545.1259613037109}}, \"EndTime\": 1604871638.780599, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871638.234858}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:38 INFO 139813082523456] #progress_metric: host=algo-1, completed 25 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 4642, \"sum\": 4642.0, \"min\": 4642}, \"Total Records Seen\": {\"count\": 1, \"max\": 4620070, \"sum\": 4620070.0, \"min\": 4620070}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 52, \"sum\": 52.0, \"min\": 52}}, \"EndTime\": 1604871638.780802, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 50}, \"StartTime\": 1604871638.235442}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:38 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=166044.287415 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:38 INFO 139813082523456] #quality_metric: host=algo-1, epoch=51, batch=0 train binary_classification_accuracy <score>=0.709\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:38 INFO 139813082523456] #quality_metric: host=algo-1, epoch=51, batch=0 train binary_classification_cross_entropy <loss>=0.556049438477\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:38 INFO 139813082523456] #quality_metric: host=algo-1, epoch=51, batch=0 train binary_f_1.000 <score>=0.739013452915\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:39.412] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 104, \"duration\": 629, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:39 INFO 139813082523456] #quality_metric: host=algo-1, epoch=51, train binary_classification_accuracy <score>=0.74256043956\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:39 INFO 139813082523456] #quality_metric: host=algo-1, epoch=51, train binary_classification_cross_entropy <loss>=0.537130896935\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:39 INFO 139813082523456] #quality_metric: host=algo-1, epoch=51, train binary_f_1.000 <score>=0.773890299105\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 631.3281059265137, \"sum\": 631.3281059265137, \"min\": 631.3281059265137}}, \"EndTime\": 1604871639.412617, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871638.780656}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:39 INFO 139813082523456] #progress_metric: host=algo-1, completed 26 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 4733, \"sum\": 4733.0, \"min\": 4733}, \"Total Records Seen\": {\"count\": 1, \"max\": 4710640, \"sum\": 4710640.0, \"min\": 4710640}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 53, \"sum\": 53.0, \"min\": 53}}, \"EndTime\": 1604871639.412804, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 51}, \"StartTime\": 1604871638.78126}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:39 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=143386.375615 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:39 INFO 139813082523456] #quality_metric: host=algo-1, epoch=52, batch=0 train binary_classification_accuracy <score>=0.708\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:39 INFO 139813082523456] #quality_metric: host=algo-1, epoch=52, batch=0 train binary_classification_cross_entropy <loss>=0.555485961914\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:39 INFO 139813082523456] #quality_metric: host=algo-1, epoch=52, batch=0 train binary_f_1.000 <score>=0.737881508079\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:39.967] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 106, \"duration\": 553, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:39 INFO 139813082523456] #quality_metric: host=algo-1, epoch=52, train binary_classification_accuracy <score>=0.742494505495\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:39 INFO 139813082523456] #quality_metric: host=algo-1, epoch=52, train binary_classification_cross_entropy <loss>=0.536416730944\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:39 INFO 139813082523456] #quality_metric: host=algo-1, epoch=52, train binary_f_1.000 <score>=0.773736300874\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 555.1600456237793, \"sum\": 555.1600456237793, \"min\": 555.1600456237793}}, \"EndTime\": 1604871639.968536, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871639.412674}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:39 INFO 139813082523456] #progress_metric: host=algo-1, completed 26 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 4824, \"sum\": 4824.0, \"min\": 4824}, \"Total Records Seen\": {\"count\": 1, \"max\": 4801210, \"sum\": 4801210.0, \"min\": 4801210}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 54, \"sum\": 54.0, \"min\": 54}}, \"EndTime\": 1604871639.96874, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 52}, \"StartTime\": 1604871639.413346}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:39 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=163045.234367 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:39 INFO 139813082523456] #quality_metric: host=algo-1, epoch=53, batch=0 train binary_classification_accuracy <score>=0.709\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:39 INFO 139813082523456] #quality_metric: host=algo-1, epoch=53, batch=0 train binary_classification_cross_entropy <loss>=0.554935974121\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:39 INFO 139813082523456] #quality_metric: host=algo-1, epoch=53, batch=0 train binary_f_1.000 <score>=0.739013452915\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:40.531] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 108, \"duration\": 560, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:40 INFO 139813082523456] #quality_metric: host=algo-1, epoch=53, train binary_classification_accuracy <score>=0.742769230769\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:40 INFO 139813082523456] #quality_metric: host=algo-1, epoch=53, train binary_classification_cross_entropy <loss>=0.535719905644\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:40 INFO 139813082523456] #quality_metric: host=algo-1, epoch=53, train binary_f_1.000 <score>=0.773944954128\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 562.72292137146, \"sum\": 562.72292137146, \"min\": 562.72292137146}}, \"EndTime\": 1604871640.531981, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871639.968594}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:40 INFO 139813082523456] #progress_metric: host=algo-1, completed 27 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 4915, \"sum\": 4915.0, \"min\": 4915}, \"Total Records Seen\": {\"count\": 1, \"max\": 4891780, \"sum\": 4891780.0, \"min\": 4891780}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 55, \"sum\": 55.0, \"min\": 55}}, \"EndTime\": 1604871640.532193, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 53}, \"StartTime\": 1604871639.969227}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:40 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=160853.182117 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:40 INFO 139813082523456] #quality_metric: host=algo-1, epoch=54, batch=0 train binary_classification_accuracy <score>=0.708\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:40 INFO 139813082523456] #quality_metric: host=algo-1, epoch=54, batch=0 train binary_classification_cross_entropy <loss>=0.554398925781\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:40 INFO 139813082523456] #quality_metric: host=algo-1, epoch=54, batch=0 train binary_f_1.000 <score>=0.737881508079\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:41.112] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 110, \"duration\": 578, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:41 INFO 139813082523456] #quality_metric: host=algo-1, epoch=54, train binary_classification_accuracy <score>=0.742978021978\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:41 INFO 139813082523456] #quality_metric: host=algo-1, epoch=54, train binary_classification_cross_entropy <loss>=0.535039755685\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:41 INFO 139813082523456] #quality_metric: host=algo-1, epoch=54, train binary_f_1.000 <score>=0.774086988438\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 580.4100036621094, \"sum\": 580.4100036621094, \"min\": 580.4100036621094}}, \"EndTime\": 1604871641.113109, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871640.532055}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:41 INFO 139813082523456] #progress_metric: host=algo-1, completed 27 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 5006, \"sum\": 5006.0, \"min\": 5006}, \"Total Records Seen\": {\"count\": 1, \"max\": 4982350, \"sum\": 4982350.0, \"min\": 4982350}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 56, \"sum\": 56.0, \"min\": 56}}, \"EndTime\": 1604871641.113301, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 54}, \"StartTime\": 1604871640.532668}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:41 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=155954.855993 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:41 INFO 139813082523456] #quality_metric: host=algo-1, epoch=55, batch=0 train binary_classification_accuracy <score>=0.708\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:41 INFO 139813082523456] #quality_metric: host=algo-1, epoch=55, batch=0 train binary_classification_cross_entropy <loss>=0.553874450684\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:41 INFO 139813082523456] #quality_metric: host=algo-1, epoch=55, batch=0 train binary_f_1.000 <score>=0.737881508079\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:41.657] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 112, \"duration\": 541, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:41 INFO 139813082523456] #quality_metric: host=algo-1, epoch=55, train binary_classification_accuracy <score>=0.743043956044\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:41 INFO 139813082523456] #quality_metric: host=algo-1, epoch=55, train binary_classification_cross_entropy <loss>=0.534375682118\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:41 INFO 139813082523456] #quality_metric: host=algo-1, epoch=55, train binary_f_1.000 <score>=0.774110032362\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 543.7939167022705, \"sum\": 543.7939167022705, \"min\": 543.7939167022705}}, \"EndTime\": 1604871641.657654, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871641.113167}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:41 INFO 139813082523456] #progress_metric: host=algo-1, completed 28 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 5097, \"sum\": 5097.0, \"min\": 5097}, \"Total Records Seen\": {\"count\": 1, \"max\": 5072920, \"sum\": 5072920.0, \"min\": 5072920}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 57, \"sum\": 57.0, \"min\": 57}}, \"EndTime\": 1604871641.657905, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 55}, \"StartTime\": 1604871641.1138}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:41 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=166429.040706 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:41 INFO 139813082523456] #quality_metric: host=algo-1, epoch=56, batch=0 train binary_classification_accuracy <score>=0.708\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:41 INFO 139813082523456] #quality_metric: host=algo-1, epoch=56, batch=0 train binary_classification_cross_entropy <loss>=0.553361999512\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:41 INFO 139813082523456] #quality_metric: host=algo-1, epoch=56, batch=0 train binary_f_1.000 <score>=0.737881508079\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:42.204] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 114, \"duration\": 545, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:42 INFO 139813082523456] #quality_metric: host=algo-1, epoch=56, train binary_classification_accuracy <score>=0.743175824176\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:42 INFO 139813082523456] #quality_metric: host=algo-1, epoch=56, train binary_classification_cross_entropy <loss>=0.533727084988\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:42 INFO 139813082523456] #quality_metric: host=algo-1, epoch=56, train binary_f_1.000 <score>=0.774230319658\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 546.6818809509277, \"sum\": 546.6818809509277, \"min\": 546.6818809509277}}, \"EndTime\": 1604871642.205106, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871641.657714}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:42 INFO 139813082523456] #progress_metric: host=algo-1, completed 28 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 5188, \"sum\": 5188.0, \"min\": 5188}, \"Total Records Seen\": {\"count\": 1, \"max\": 5163490, \"sum\": 5163490.0, \"min\": 5163490}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 58, \"sum\": 58.0, \"min\": 58}}, \"EndTime\": 1604871642.205306, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 56}, \"StartTime\": 1604871641.658398}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:42 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=165571.053984 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:42 INFO 139813082523456] #quality_metric: host=algo-1, epoch=57, batch=0 train binary_classification_accuracy <score>=0.708\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:42 INFO 139813082523456] #quality_metric: host=algo-1, epoch=57, batch=0 train binary_classification_cross_entropy <loss>=0.552861083984\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:42 INFO 139813082523456] #quality_metric: host=algo-1, epoch=57, batch=0 train binary_f_1.000 <score>=0.737881508079\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:42.767] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 116, \"duration\": 560, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:42 INFO 139813082523456] #quality_metric: host=algo-1, epoch=57, train binary_classification_accuracy <score>=0.743120879121\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:42 INFO 139813082523456] #quality_metric: host=algo-1, epoch=57, train binary_classification_cross_entropy <loss>=0.53309341263\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:42 INFO 139813082523456] #quality_metric: host=algo-1, epoch=57, train binary_f_1.000 <score>=0.774149291801\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 562.5278949737549, \"sum\": 562.5278949737549, \"min\": 562.5278949737549}}, \"EndTime\": 1604871642.768343, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871642.205166}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:42 INFO 139813082523456] #progress_metric: host=algo-1, completed 29 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 5279, \"sum\": 5279.0, \"min\": 5279}, \"Total Records Seen\": {\"count\": 1, \"max\": 5254060, \"sum\": 5254060.0, \"min\": 5254060}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 59, \"sum\": 59.0, \"min\": 59}}, \"EndTime\": 1604871642.768496, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 57}, \"StartTime\": 1604871642.205789}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:42 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=160930.115822 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:42 INFO 139813082523456] #quality_metric: host=algo-1, epoch=58, batch=0 train binary_classification_accuracy <score>=0.709\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:42 INFO 139813082523456] #quality_metric: host=algo-1, epoch=58, batch=0 train binary_classification_cross_entropy <loss>=0.552371337891\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:42 INFO 139813082523456] #quality_metric: host=algo-1, epoch=58, batch=0 train binary_f_1.000 <score>=0.738544474394\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:43.328] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 118, \"duration\": 558, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:43 INFO 139813082523456] #quality_metric: host=algo-1, epoch=58, train binary_classification_accuracy <score>=0.743296703297\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:43 INFO 139813082523456] #quality_metric: host=algo-1, epoch=58, train binary_classification_cross_entropy <loss>=0.532474125789\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:43 INFO 139813082523456] #quality_metric: host=algo-1, epoch=58, train binary_f_1.000 <score>=0.774268983244\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 559.9820613861084, \"sum\": 559.9820613861084, \"min\": 559.9820613861084}}, \"EndTime\": 1604871643.328977, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871642.7684}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:43 INFO 139813082523456] #progress_metric: host=algo-1, completed 29 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 5370, \"sum\": 5370.0, \"min\": 5370}, \"Total Records Seen\": {\"count\": 1, \"max\": 5344630, \"sum\": 5344630.0, \"min\": 5344630}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 60, \"sum\": 60.0, \"min\": 60}}, \"EndTime\": 1604871643.329189, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 58}, \"StartTime\": 1604871642.768963}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:43 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=161634.028787 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:43 INFO 139813082523456] #quality_metric: host=algo-1, epoch=59, batch=0 train binary_classification_accuracy <score>=0.709\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:43 INFO 139813082523456] #quality_metric: host=algo-1, epoch=59, batch=0 train binary_classification_cross_entropy <loss>=0.551892333984\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:43 INFO 139813082523456] #quality_metric: host=algo-1, epoch=59, batch=0 train binary_f_1.000 <score>=0.738544474394\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:43.896] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 120, \"duration\": 565, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:43 INFO 139813082523456] #quality_metric: host=algo-1, epoch=59, train binary_classification_accuracy <score>=0.743296703297\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:43 INFO 139813082523456] #quality_metric: host=algo-1, epoch=59, train binary_classification_cross_entropy <loss>=0.531868693928\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:43 INFO 139813082523456] #quality_metric: host=algo-1, epoch=59, train binary_f_1.000 <score>=0.774212255944\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 567.4121379852295, \"sum\": 567.4121379852295, \"min\": 567.4121379852295}}, \"EndTime\": 1604871643.897151, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871643.329036}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:43 INFO 139813082523456] #progress_metric: host=algo-1, completed 30 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 5461, \"sum\": 5461.0, \"min\": 5461}, \"Total Records Seen\": {\"count\": 1, \"max\": 5435200, \"sum\": 5435200.0, \"min\": 5435200}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 61, \"sum\": 61.0, \"min\": 61}}, \"EndTime\": 1604871643.897363, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 59}, \"StartTime\": 1604871643.32971}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:43 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=159523.306182 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:43 INFO 139813082523456] #quality_metric: host=algo-1, epoch=60, batch=0 train binary_classification_accuracy <score>=0.711\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:43 INFO 139813082523456] #quality_metric: host=algo-1, epoch=60, batch=0 train binary_classification_cross_entropy <loss>=0.551423583984\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:43 INFO 139813082523456] #quality_metric: host=algo-1, epoch=60, batch=0 train binary_f_1.000 <score>=0.740341419587\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:44.491] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 122, \"duration\": 592, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:44 INFO 139813082523456] #quality_metric: host=algo-1, epoch=60, train binary_classification_accuracy <score>=0.743395604396\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:44 INFO 139813082523456] #quality_metric: host=algo-1, epoch=60, train binary_classification_cross_entropy <loss>=0.531276642852\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:44 INFO 139813082523456] #quality_metric: host=algo-1, epoch=60, train binary_f_1.000 <score>=0.774253424724\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 594.4130420684814, \"sum\": 594.4130420684814, \"min\": 594.4130420684814}}, \"EndTime\": 1604871644.49227, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871643.897214}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:44 INFO 139813082523456] #progress_metric: host=algo-1, completed 30 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 5552, \"sum\": 5552.0, \"min\": 5552}, \"Total Records Seen\": {\"count\": 1, \"max\": 5525770, \"sum\": 5525770.0, \"min\": 5525770}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 62, \"sum\": 62.0, \"min\": 62}}, \"EndTime\": 1604871644.492449, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 60}, \"StartTime\": 1604871643.897831}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:44 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=152293.479218 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:44 INFO 139813082523456] #quality_metric: host=algo-1, epoch=61, batch=0 train binary_classification_accuracy <score>=0.711\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:44 INFO 139813082523456] #quality_metric: host=algo-1, epoch=61, batch=0 train binary_classification_cross_entropy <loss>=0.55096484375\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:44 INFO 139813082523456] #quality_metric: host=algo-1, epoch=61, batch=0 train binary_f_1.000 <score>=0.740341419587\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:45.056] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 124, \"duration\": 562, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:45 INFO 139813082523456] #quality_metric: host=algo-1, epoch=61, train binary_classification_accuracy <score>=0.743406593407\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:45 INFO 139813082523456] #quality_metric: host=algo-1, epoch=61, train binary_classification_cross_entropy <loss>=0.530697477571\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:45 INFO 139813082523456] #quality_metric: host=algo-1, epoch=61, train binary_f_1.000 <score>=0.77421725425\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 564.0718936920166, \"sum\": 564.0718936920166, \"min\": 564.0718936920166}}, \"EndTime\": 1604871645.057032, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871644.492326}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:45 INFO 139813082523456] #progress_metric: host=algo-1, completed 31 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 5643, \"sum\": 5643.0, \"min\": 5643}, \"Total Records Seen\": {\"count\": 1, \"max\": 5616340, \"sum\": 5616340.0, \"min\": 5616340}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 63, \"sum\": 63.0, \"min\": 63}}, \"EndTime\": 1604871645.057258, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 61}, \"StartTime\": 1604871644.49293}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:45 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=160461.344827 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:45 INFO 139813082523456] #quality_metric: host=algo-1, epoch=62, batch=0 train binary_classification_accuracy <score>=0.712\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:45 INFO 139813082523456] #quality_metric: host=algo-1, epoch=62, batch=0 train binary_classification_cross_entropy <loss>=0.550515563965\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:45 INFO 139813082523456] #quality_metric: host=algo-1, epoch=62, batch=0 train binary_f_1.000 <score>=0.741007194245\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:45.606] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 126, \"duration\": 547, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:45 INFO 139813082523456] #quality_metric: host=algo-1, epoch=62, train binary_classification_accuracy <score>=0.743516483516\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:45 INFO 139813082523456] #quality_metric: host=algo-1, epoch=62, train binary_classification_cross_entropy <loss>=0.530130751725\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:45 INFO 139813082523456] #quality_metric: host=algo-1, epoch=62, train binary_f_1.000 <score>=0.774296489701\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 549.7629642486572, \"sum\": 549.7629642486572, \"min\": 549.7629642486572}}, \"EndTime\": 1604871645.607556, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871645.057098}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:45 INFO 139813082523456] #progress_metric: host=algo-1, completed 31 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 5734, \"sum\": 5734.0, \"min\": 5734}, \"Total Records Seen\": {\"count\": 1, \"max\": 5706910, \"sum\": 5706910.0, \"min\": 5706910}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 64, \"sum\": 64.0, \"min\": 64}}, \"EndTime\": 1604871645.607752, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 62}, \"StartTime\": 1604871645.057767}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:45 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=164644.906606 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:45 INFO 139813082523456] #quality_metric: host=algo-1, epoch=63, batch=0 train binary_classification_accuracy <score>=0.712\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:45 INFO 139813082523456] #quality_metric: host=algo-1, epoch=63, batch=0 train binary_classification_cross_entropy <loss>=0.550075500488\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:45 INFO 139813082523456] #quality_metric: host=algo-1, epoch=63, batch=0 train binary_f_1.000 <score>=0.741472172352\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:46.161] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 128, \"duration\": 551, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:46 INFO 139813082523456] #quality_metric: host=algo-1, epoch=63, train binary_classification_accuracy <score>=0.743769230769\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:46 INFO 139813082523456] #quality_metric: host=algo-1, epoch=63, train binary_classification_cross_entropy <loss>=0.529576001848\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:46 INFO 139813082523456] #quality_metric: host=algo-1, epoch=63, train binary_f_1.000 <score>=0.774477469025\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 553.6391735076904, \"sum\": 553.6391735076904, \"min\": 553.6391735076904}}, \"EndTime\": 1604871646.161941, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871645.607618}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:46 INFO 139813082523456] #progress_metric: host=algo-1, completed 32 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 5825, \"sum\": 5825.0, \"min\": 5825}, \"Total Records Seen\": {\"count\": 1, \"max\": 5797480, \"sum\": 5797480.0, \"min\": 5797480}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 65, \"sum\": 65.0, \"min\": 65}}, \"EndTime\": 1604871646.162118, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 63}, \"StartTime\": 1604871645.608275}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:46 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=163503.837883 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:46 INFO 139813082523456] #quality_metric: host=algo-1, epoch=64, batch=0 train binary_classification_accuracy <score>=0.711\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:46 INFO 139813082523456] #quality_metric: host=algo-1, epoch=64, batch=0 train binary_classification_cross_entropy <loss>=0.549644165039\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:46 INFO 139813082523456] #quality_metric: host=algo-1, epoch=64, batch=0 train binary_f_1.000 <score>=0.740341419587\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:46.725] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 130, \"duration\": 561, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:46 INFO 139813082523456] #quality_metric: host=algo-1, epoch=64, train binary_classification_accuracy <score>=0.743923076923\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:46 INFO 139813082523456] #quality_metric: host=algo-1, epoch=64, train binary_classification_cross_entropy <loss>=0.529032814445\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:46 INFO 139813082523456] #quality_metric: host=algo-1, epoch=64, train binary_f_1.000 <score>=0.774582353909\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 563.2669925689697, \"sum\": 563.2669925689697, \"min\": 563.2669925689697}}, \"EndTime\": 1604871646.725897, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871646.162004}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:46 INFO 139813082523456] #progress_metric: host=algo-1, completed 32 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 5916, \"sum\": 5916.0, \"min\": 5916}, \"Total Records Seen\": {\"count\": 1, \"max\": 5888050, \"sum\": 5888050.0, \"min\": 5888050}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 66, \"sum\": 66.0, \"min\": 66}}, \"EndTime\": 1604871646.72625, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 64}, \"StartTime\": 1604871646.162592}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:46 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=160654.612554 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:46 INFO 139813082523456] #quality_metric: host=algo-1, epoch=65, batch=0 train binary_classification_accuracy <score>=0.712\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:46 INFO 139813082523456] #quality_metric: host=algo-1, epoch=65, batch=0 train binary_classification_cross_entropy <loss>=0.549221313477\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:46 INFO 139813082523456] #quality_metric: host=algo-1, epoch=65, batch=0 train binary_f_1.000 <score>=0.741472172352\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:47.300] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 132, \"duration\": 572, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:47 INFO 139813082523456] #quality_metric: host=algo-1, epoch=65, train binary_classification_accuracy <score>=0.744153846154\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:47 INFO 139813082523456] #quality_metric: host=algo-1, epoch=65, train binary_classification_cross_entropy <loss>=0.528500771994\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:47 INFO 139813082523456] #quality_metric: host=algo-1, epoch=65, train binary_f_1.000 <score>=0.77475716884\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 574.4788646697998, \"sum\": 574.4788646697998, \"min\": 574.4788646697998}}, \"EndTime\": 1604871647.3013, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871646.725963}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:47 INFO 139813082523456] #progress_metric: host=algo-1, completed 33 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 6007, \"sum\": 6007.0, \"min\": 6007}, \"Total Records Seen\": {\"count\": 1, \"max\": 5978620, \"sum\": 5978620.0, \"min\": 5978620}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 67, \"sum\": 67.0, \"min\": 67}}, \"EndTime\": 1604871647.301513, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 65}, \"StartTime\": 1604871646.726791}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:47 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=157559.611184 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:47 INFO 139813082523456] #quality_metric: host=algo-1, epoch=66, batch=0 train binary_classification_accuracy <score>=0.712\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:47 INFO 139813082523456] #quality_metric: host=algo-1, epoch=66, batch=0 train binary_classification_cross_entropy <loss>=0.548806640625\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:47 INFO 139813082523456] #quality_metric: host=algo-1, epoch=66, batch=0 train binary_f_1.000 <score>=0.741472172352\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:47.861] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 134, \"duration\": 558, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:47 INFO 139813082523456] #quality_metric: host=algo-1, epoch=66, train binary_classification_accuracy <score>=0.74432967033\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:47 INFO 139813082523456] #quality_metric: host=algo-1, epoch=66, train binary_classification_cross_entropy <loss>=0.527979460664\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:47 INFO 139813082523456] #quality_metric: host=algo-1, epoch=66, train binary_f_1.000 <score>=0.774890183254\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 559.9181652069092, \"sum\": 559.9181652069092, \"min\": 559.9181652069092}}, \"EndTime\": 1604871647.86197, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871647.30136}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:47 INFO 139813082523456] #progress_metric: host=algo-1, completed 33 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 6098, \"sum\": 6098.0, \"min\": 6098}, \"Total Records Seen\": {\"count\": 1, \"max\": 6069190, \"sum\": 6069190.0, \"min\": 6069190}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 68, \"sum\": 68.0, \"min\": 68}}, \"EndTime\": 1604871647.862182, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 66}, \"StartTime\": 1604871647.302027}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:47 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=161653.975572 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:47 INFO 139813082523456] #quality_metric: host=algo-1, epoch=67, batch=0 train binary_classification_accuracy <score>=0.712\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:47 INFO 139813082523456] #quality_metric: host=algo-1, epoch=67, batch=0 train binary_classification_cross_entropy <loss>=0.548399780273\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:47 INFO 139813082523456] #quality_metric: host=algo-1, epoch=67, batch=0 train binary_f_1.000 <score>=0.741472172352\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[34m[2020-11-08 21:40:48.425] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 136, \"duration\": 561, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:48 INFO 139813082523456] #quality_metric: host=algo-1, epoch=67, train binary_classification_accuracy <score>=0.744406593407\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:48 INFO 139813082523456] #quality_metric: host=algo-1, epoch=67, train binary_classification_cross_entropy <loss>=0.527468503512\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:48 INFO 139813082523456] #quality_metric: host=algo-1, epoch=67, train binary_f_1.000 <score>=0.774933957791\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 563.0748271942139, \"sum\": 563.0748271942139, \"min\": 563.0748271942139}}, \"EndTime\": 1604871648.42579, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871647.862052}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:48 INFO 139813082523456] #progress_metric: host=algo-1, completed 34 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 6189, \"sum\": 6189.0, \"min\": 6189}, \"Total Records Seen\": {\"count\": 1, \"max\": 6159760, \"sum\": 6159760.0, \"min\": 6159760}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 69, \"sum\": 69.0, \"min\": 69}}, \"EndTime\": 1604871648.426032, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 67}, \"StartTime\": 1604871647.862686}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:48 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=160739.31366 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:48 INFO 139813082523456] #quality_metric: host=algo-1, epoch=68, batch=0 train binary_classification_accuracy <score>=0.713\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:48 INFO 139813082523456] #quality_metric: host=algo-1, epoch=68, batch=0 train binary_classification_cross_entropy <loss>=0.548000305176\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:48 INFO 139813082523456] #quality_metric: host=algo-1, epoch=68, batch=0 train binary_f_1.000 <score>=0.74213836478\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:48.991] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 138, \"duration\": 564, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:48 INFO 139813082523456] #quality_metric: host=algo-1, epoch=68, train binary_classification_accuracy <score>=0.744549450549\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:48 INFO 139813082523456] #quality_metric: host=algo-1, epoch=68, train binary_classification_cross_entropy <loss>=0.526967505822\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:48 INFO 139813082523456] #quality_metric: host=algo-1, epoch=68, train binary_f_1.000 <score>=0.775027098173\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 565.7150745391846, \"sum\": 565.7150745391846, \"min\": 565.7150745391846}}, \"EndTime\": 1604871648.992286, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871648.425861}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:48 INFO 139813082523456] #progress_metric: host=algo-1, completed 34 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 6280, \"sum\": 6280.0, \"min\": 6280}, \"Total Records Seen\": {\"count\": 1, \"max\": 6250330, \"sum\": 6250330.0, \"min\": 6250330}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 70, \"sum\": 70.0, \"min\": 70}}, \"EndTime\": 1604871648.992481, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 68}, \"StartTime\": 1604871648.426541}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:48 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=160008.033835 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:49 INFO 139813082523456] #quality_metric: host=algo-1, epoch=69, batch=0 train binary_classification_accuracy <score>=0.712\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:49 INFO 139813082523456] #quality_metric: host=algo-1, epoch=69, batch=0 train binary_classification_cross_entropy <loss>=0.547608032227\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:49 INFO 139813082523456] #quality_metric: host=algo-1, epoch=69, batch=0 train binary_f_1.000 <score>=0.741472172352\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:49.637] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 140, \"duration\": 643, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:49 INFO 139813082523456] #quality_metric: host=algo-1, epoch=69, train binary_classification_accuracy <score>=0.744835164835\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:49 INFO 139813082523456] #quality_metric: host=algo-1, epoch=69, train binary_classification_cross_entropy <loss>=0.526476116474\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:49 INFO 139813082523456] #quality_metric: host=algo-1, epoch=69, train binary_f_1.000 <score>=0.775283073648\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 645.3659534454346, \"sum\": 645.3659534454346, \"min\": 645.3659534454346}}, \"EndTime\": 1604871649.63834, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871648.992351}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:49 INFO 139813082523456] #progress_metric: host=algo-1, completed 35 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 6371, \"sum\": 6371.0, \"min\": 6371}, \"Total Records Seen\": {\"count\": 1, \"max\": 6340900, \"sum\": 6340900.0, \"min\": 6340900}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 71, \"sum\": 71.0, \"min\": 71}}, \"EndTime\": 1604871649.638509, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 69}, \"StartTime\": 1604871648.992945}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:49 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=140274.617797 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:49 INFO 139813082523456] #quality_metric: host=algo-1, epoch=70, batch=0 train binary_classification_accuracy <score>=0.712\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:49 INFO 139813082523456] #quality_metric: host=algo-1, epoch=70, batch=0 train binary_classification_cross_entropy <loss>=0.547222595215\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:49 INFO 139813082523456] #quality_metric: host=algo-1, epoch=70, batch=0 train binary_f_1.000 <score>=0.741472172352\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:50.204] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 142, \"duration\": 563, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:50 INFO 139813082523456] #quality_metric: host=algo-1, epoch=70, train binary_classification_accuracy <score>=0.744923076923\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:50 INFO 139813082523456] #quality_metric: host=algo-1, epoch=70, train binary_classification_cross_entropy <loss>=0.525993978312\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:50 INFO 139813082523456] #quality_metric: host=algo-1, epoch=70, train binary_f_1.000 <score>=0.7753257061\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 565.4940605163574, \"sum\": 565.4940605163574, \"min\": 565.4940605163574}}, \"EndTime\": 1604871650.204534, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871649.638391}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:50 INFO 139813082523456] #progress_metric: host=algo-1, completed 35 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 6462, \"sum\": 6462.0, \"min\": 6462}, \"Total Records Seen\": {\"count\": 1, \"max\": 6431470, \"sum\": 6431470.0, \"min\": 6431470}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 72, \"sum\": 72.0, \"min\": 72}}, \"EndTime\": 1604871650.204689, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 70}, \"StartTime\": 1604871649.639014}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:50 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=160085.780108 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:50 INFO 139813082523456] #quality_metric: host=algo-1, epoch=71, batch=0 train binary_classification_accuracy <score>=0.712\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:50 INFO 139813082523456] #quality_metric: host=algo-1, epoch=71, batch=0 train binary_classification_cross_entropy <loss>=0.546843688965\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:50 INFO 139813082523456] #quality_metric: host=algo-1, epoch=71, batch=0 train binary_f_1.000 <score>=0.741472172352\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:50.780] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 144, \"duration\": 574, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:50 INFO 139813082523456] #quality_metric: host=algo-1, epoch=71, train binary_classification_accuracy <score>=0.745164835165\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:50 INFO 139813082523456] #quality_metric: host=algo-1, epoch=71, train binary_classification_cross_entropy <loss>=0.525520734515\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:50 INFO 139813082523456] #quality_metric: host=algo-1, epoch=71, train binary_f_1.000 <score>=0.775560373195\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 575.7198333740234, \"sum\": 575.7198333740234, \"min\": 575.7198333740234}}, \"EndTime\": 1604871650.780908, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871650.204593}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:50 INFO 139813082523456] #progress_metric: host=algo-1, completed 36 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 6553, \"sum\": 6553.0, \"min\": 6553}, \"Total Records Seen\": {\"count\": 1, \"max\": 6522040, \"sum\": 6522040.0, \"min\": 6522040}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 73, \"sum\": 73.0, \"min\": 73}}, \"EndTime\": 1604871650.781118, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 71}, \"StartTime\": 1604871650.205159}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:50 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=157221.238332 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:50 INFO 139813082523456] #quality_metric: host=algo-1, epoch=72, batch=0 train binary_classification_accuracy <score>=0.713\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:50 INFO 139813082523456] #quality_metric: host=algo-1, epoch=72, batch=0 train binary_classification_cross_entropy <loss>=0.546471069336\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:50 INFO 139813082523456] #quality_metric: host=algo-1, epoch=72, batch=0 train binary_f_1.000 <score>=0.74213836478\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:51.319] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 146, \"duration\": 537, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:51 INFO 139813082523456] #quality_metric: host=algo-1, epoch=72, train binary_classification_accuracy <score>=0.745483516484\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:51 INFO 139813082523456] #quality_metric: host=algo-1, epoch=72, train binary_classification_cross_entropy <loss>=0.525056064145\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:51 INFO 139813082523456] #quality_metric: host=algo-1, epoch=72, train binary_f_1.000 <score>=0.775817176928\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 538.517951965332, \"sum\": 538.517951965332, \"min\": 538.517951965332}}, \"EndTime\": 1604871651.320148, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871650.780979}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:51 INFO 139813082523456] #progress_metric: host=algo-1, completed 36 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 6644, \"sum\": 6644.0, \"min\": 6644}, \"Total Records Seen\": {\"count\": 1, \"max\": 6612610, \"sum\": 6612610.0, \"min\": 6612610}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 74, \"sum\": 74.0, \"min\": 74}}, \"EndTime\": 1604871651.320299, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 72}, \"StartTime\": 1604871650.781605}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:51 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=168100.751775 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:51 INFO 139813082523456] #quality_metric: host=algo-1, epoch=73, batch=0 train binary_classification_accuracy <score>=0.713\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:51 INFO 139813082523456] #quality_metric: host=algo-1, epoch=73, batch=0 train binary_classification_cross_entropy <loss>=0.546104492188\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:51 INFO 139813082523456] #quality_metric: host=algo-1, epoch=73, batch=0 train binary_f_1.000 <score>=0.74213836478\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:51.870] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 148, \"duration\": 548, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:51 INFO 139813082523456] #quality_metric: host=algo-1, epoch=73, train binary_classification_accuracy <score>=0.745582417582\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:51 INFO 139813082523456] #quality_metric: host=algo-1, epoch=73, train binary_classification_cross_entropy <loss>=0.524599628826\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:51 INFO 139813082523456] #quality_metric: host=algo-1, epoch=73, train binary_f_1.000 <score>=0.77591079794\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 549.7329235076904, \"sum\": 549.7329235076904, \"min\": 549.7329235076904}}, \"EndTime\": 1604871651.870552, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871651.320205}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:51 INFO 139813082523456] #progress_metric: host=algo-1, completed 37 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 6735, \"sum\": 6735.0, \"min\": 6735}, \"Total Records Seen\": {\"count\": 1, \"max\": 6703180, \"sum\": 6703180.0, \"min\": 6703180}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 75, \"sum\": 75.0, \"min\": 75}}, \"EndTime\": 1604871651.870706, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 73}, \"StartTime\": 1604871651.320792}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:51 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=164673.312621 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:51 INFO 139813082523456] #quality_metric: host=algo-1, epoch=74, batch=0 train binary_classification_accuracy <score>=0.713\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:51 INFO 139813082523456] #quality_metric: host=algo-1, epoch=74, batch=0 train binary_classification_cross_entropy <loss>=0.545743530273\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:51 INFO 139813082523456] #quality_metric: host=algo-1, epoch=74, batch=0 train binary_f_1.000 <score>=0.74213836478\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:52.425] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 150, \"duration\": 552, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:52 INFO 139813082523456] #quality_metric: host=algo-1, epoch=74, train binary_classification_accuracy <score>=0.745791208791\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:52 INFO 139813082523456] #quality_metric: host=algo-1, epoch=74, train binary_classification_cross_entropy <loss>=0.524151119358\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:52 INFO 139813082523456] #quality_metric: host=algo-1, epoch=74, train binary_f_1.000 <score>=0.776079528405\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 554.2941093444824, \"sum\": 554.2941093444824, \"min\": 554.2941093444824}}, \"EndTime\": 1604871652.425523, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871651.870612}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:52 INFO 139813082523456] #progress_metric: host=algo-1, completed 37 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 6826, \"sum\": 6826.0, \"min\": 6826}, \"Total Records Seen\": {\"count\": 1, \"max\": 6793750, \"sum\": 6793750.0, \"min\": 6793750}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 76, \"sum\": 76.0, \"min\": 76}}, \"EndTime\": 1604871652.425745, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 74}, \"StartTime\": 1604871651.8712}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:52 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=163290.391352 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:52 INFO 139813082523456] #quality_metric: host=algo-1, epoch=75, batch=0 train binary_classification_accuracy <score>=0.712\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:52 INFO 139813082523456] #quality_metric: host=algo-1, epoch=75, batch=0 train binary_classification_cross_entropy <loss>=0.545388061523\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:52 INFO 139813082523456] #quality_metric: host=algo-1, epoch=75, batch=0 train binary_f_1.000 <score>=0.741007194245\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:53.002] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 152, \"duration\": 574, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:53 INFO 139813082523456] #quality_metric: host=algo-1, epoch=75, train binary_classification_accuracy <score>=0.745967032967\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:53 INFO 139813082523456] #quality_metric: host=algo-1, epoch=75, train binary_classification_cross_entropy <loss>=0.523710223187\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:53 INFO 139813082523456] #quality_metric: host=algo-1, epoch=75, train binary_f_1.000 <score>=0.776225739316\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 576.5979290008545, \"sum\": 576.5979290008545, \"min\": 576.5979290008545}}, \"EndTime\": 1604871653.002883, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871652.425598}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:53 INFO 139813082523456] #progress_metric: host=algo-1, completed 38 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 6917, \"sum\": 6917.0, \"min\": 6917}, \"Total Records Seen\": {\"count\": 1, \"max\": 6884320, \"sum\": 6884320.0, \"min\": 6884320}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 77, \"sum\": 77.0, \"min\": 77}}, \"EndTime\": 1604871653.003186, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 75}, \"StartTime\": 1604871652.426253}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:53 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=156948.549921 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:53 INFO 139813082523456] #quality_metric: host=algo-1, epoch=76, batch=0 train binary_classification_accuracy <score>=0.713\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:53 INFO 139813082523456] #quality_metric: host=algo-1, epoch=76, batch=0 train binary_classification_cross_entropy <loss>=0.545037841797\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:53 INFO 139813082523456] #quality_metric: host=algo-1, epoch=76, batch=0 train binary_f_1.000 <score>=0.74213836478\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:53.581] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 154, \"duration\": 576, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:53 INFO 139813082523456] #quality_metric: host=algo-1, epoch=76, train binary_classification_accuracy <score>=0.746120879121\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:53 INFO 139813082523456] #quality_metric: host=algo-1, epoch=76, train binary_classification_cross_entropy <loss>=0.52327665526\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:53 INFO 139813082523456] #quality_metric: host=algo-1, epoch=76, train binary_f_1.000 <score>=0.776343940289\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 578.2740116119385, \"sum\": 578.2740116119385, \"min\": 578.2740116119385}}, \"EndTime\": 1604871653.582098, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871653.002958}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:53 INFO 139813082523456] #progress_metric: host=algo-1, completed 38 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 7008, \"sum\": 7008.0, \"min\": 7008}, \"Total Records Seen\": {\"count\": 1, \"max\": 6974890, \"sum\": 6974890.0, \"min\": 6974890}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 78, \"sum\": 78.0, \"min\": 78}}, \"EndTime\": 1604871653.582299, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 76}, \"StartTime\": 1604871653.003797}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:53 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=156534.642251 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:53 INFO 139813082523456] #quality_metric: host=algo-1, epoch=77, batch=0 train binary_classification_accuracy <score>=0.714\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:53 INFO 139813082523456] #quality_metric: host=algo-1, epoch=77, batch=0 train binary_classification_cross_entropy <loss>=0.544692504883\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:53 INFO 139813082523456] #quality_metric: host=algo-1, epoch=77, batch=0 train binary_f_1.000 <score>=0.742805755396\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:54.150] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 156, \"duration\": 566, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:54 INFO 139813082523456] #quality_metric: host=algo-1, epoch=77, train binary_classification_accuracy <score>=0.746296703297\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:54 INFO 139813082523456] #quality_metric: host=algo-1, epoch=77, train binary_classification_cross_entropy <loss>=0.522850111406\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:54 INFO 139813082523456] #quality_metric: host=algo-1, epoch=77, train binary_f_1.000 <score>=0.776481522718\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 568.0029392242432, \"sum\": 568.0029392242432, \"min\": 568.0029392242432}}, \"EndTime\": 1604871654.150797, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871653.582159}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:54 INFO 139813082523456] #progress_metric: host=algo-1, completed 39 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 7099, \"sum\": 7099.0, \"min\": 7099}, \"Total Records Seen\": {\"count\": 1, \"max\": 7065460, \"sum\": 7065460.0, \"min\": 7065460}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 79, \"sum\": 79.0, \"min\": 79}}, \"EndTime\": 1604871654.150993, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 77}, \"StartTime\": 1604871653.582762}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:54 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=159365.034195 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:54 INFO 139813082523456] #quality_metric: host=algo-1, epoch=78, batch=0 train binary_classification_accuracy <score>=0.713\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:54 INFO 139813082523456] #quality_metric: host=algo-1, epoch=78, batch=0 train binary_classification_cross_entropy <loss>=0.544351867676\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:54 INFO 139813082523456] #quality_metric: host=algo-1, epoch=78, batch=0 train binary_f_1.000 <score>=0.741674167417\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:54.731] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 158, \"duration\": 579, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:54 INFO 139813082523456] #quality_metric: host=algo-1, epoch=78, train binary_classification_accuracy <score>=0.746428571429\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:54 INFO 139813082523456] #quality_metric: host=algo-1, epoch=78, train binary_classification_cross_entropy <loss>=0.522430315961\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:54 INFO 139813082523456] #quality_metric: host=algo-1, epoch=78, train binary_f_1.000 <score>=0.776593375739\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 580.8219909667969, \"sum\": 580.8219909667969, \"min\": 580.8219909667969}}, \"EndTime\": 1604871654.7323, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871654.150856}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:54 INFO 139813082523456] #progress_metric: host=algo-1, completed 39 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 7190, \"sum\": 7190.0, \"min\": 7190}, \"Total Records Seen\": {\"count\": 1, \"max\": 7156030, \"sum\": 7156030.0, \"min\": 7156030}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 80, \"sum\": 80.0, \"min\": 80}}, \"EndTime\": 1604871654.732456, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 78}, \"StartTime\": 1604871654.151452}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:54 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=155862.010182 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:54 INFO 139813082523456] #quality_metric: host=algo-1, epoch=79, batch=0 train binary_classification_accuracy <score>=0.714\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:54 INFO 139813082523456] #quality_metric: host=algo-1, epoch=79, batch=0 train binary_classification_cross_entropy <loss>=0.544015625\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:54 INFO 139813082523456] #quality_metric: host=algo-1, epoch=79, batch=0 train binary_f_1.000 <score>=0.742342342342\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:55.297] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 160, \"duration\": 563, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:55 INFO 139813082523456] #quality_metric: host=algo-1, epoch=79, train binary_classification_accuracy <score>=0.746549450549\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:55 INFO 139813082523456] #quality_metric: host=algo-1, epoch=79, train binary_classification_cross_entropy <loss>=0.522016990913\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:55 INFO 139813082523456] #quality_metric: host=algo-1, epoch=79, train binary_f_1.000 <score>=0.776697713146\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 564.7399425506592, \"sum\": 564.7399425506592, \"min\": 564.7399425506592}}, \"EndTime\": 1604871655.297707, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871654.732355}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:55 INFO 139813082523456] #progress_metric: host=algo-1, completed 40 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 7281, \"sum\": 7281.0, \"min\": 7281}, \"Total Records Seen\": {\"count\": 1, \"max\": 7246600, \"sum\": 7246600.0, \"min\": 7246600}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 81, \"sum\": 81.0, \"min\": 81}}, \"EndTime\": 1604871655.297936, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 79}, \"StartTime\": 1604871654.732938}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:55 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=160272.801493 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:55 INFO 139813082523456] #quality_metric: host=algo-1, epoch=80, batch=0 train binary_classification_accuracy <score>=0.714\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:55 INFO 139813082523456] #quality_metric: host=algo-1, epoch=80, batch=0 train binary_classification_cross_entropy <loss>=0.54368371582\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:55 INFO 139813082523456] #quality_metric: host=algo-1, epoch=80, batch=0 train binary_f_1.000 <score>=0.742342342342\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:55.842] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 162, \"duration\": 543, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:55 INFO 139813082523456] #quality_metric: host=algo-1, epoch=80, train binary_classification_accuracy <score>=0.746637362637\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:55 INFO 139813082523456] #quality_metric: host=algo-1, epoch=80, train binary_classification_cross_entropy <loss>=0.521609887762\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:55 INFO 139813082523456] #quality_metric: host=algo-1, epoch=80, train binary_f_1.000 <score>=0.776762199845\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 544.680118560791, \"sum\": 544.680118560791, \"min\": 544.680118560791}}, \"EndTime\": 1604871655.843172, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871655.297773}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:55 INFO 139813082523456] #progress_metric: host=algo-1, completed 40 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 7372, \"sum\": 7372.0, \"min\": 7372}, \"Total Records Seen\": {\"count\": 1, \"max\": 7337170, \"sum\": 7337170.0, \"min\": 7337170}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 82, \"sum\": 82.0, \"min\": 82}}, \"EndTime\": 1604871655.843334, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 80}, \"StartTime\": 1604871655.298465}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:55 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=166196.913468 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:55 INFO 139813082523456] #quality_metric: host=algo-1, epoch=81, batch=0 train binary_classification_accuracy <score>=0.713\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:55 INFO 139813082523456] #quality_metric: host=algo-1, epoch=81, batch=0 train binary_classification_cross_entropy <loss>=0.543355712891\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:55 INFO 139813082523456] #quality_metric: host=algo-1, epoch=81, batch=0 train binary_f_1.000 <score>=0.741674167417\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:56.397] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 164, \"duration\": 552, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:56 INFO 139813082523456] #quality_metric: host=algo-1, epoch=81, train binary_classification_accuracy <score>=0.746802197802\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:56 INFO 139813082523456] #quality_metric: host=algo-1, epoch=81, train binary_classification_cross_entropy <loss>=0.521208732521\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:56 INFO 139813082523456] #quality_metric: host=algo-1, epoch=81, train binary_f_1.000 <score>=0.776887993725\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 554.0771484375, \"sum\": 554.0771484375, \"min\": 554.0771484375}}, \"EndTime\": 1604871656.397908, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871655.843231}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:56 INFO 139813082523456] #progress_metric: host=algo-1, completed 41 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 7463, \"sum\": 7463.0, \"min\": 7463}, \"Total Records Seen\": {\"count\": 1, \"max\": 7427740, \"sum\": 7427740.0, \"min\": 7427740}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 83, \"sum\": 83.0, \"min\": 83}}, \"EndTime\": 1604871656.398078, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 81}, \"StartTime\": 1604871655.843807}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:56 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=163377.333188 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:56 INFO 139813082523456] #quality_metric: host=algo-1, epoch=82, batch=0 train binary_classification_accuracy <score>=0.712\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:56 INFO 139813082523456] #quality_metric: host=algo-1, epoch=82, batch=0 train binary_classification_cross_entropy <loss>=0.543031555176\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:56 INFO 139813082523456] #quality_metric: host=algo-1, epoch=82, batch=0 train binary_f_1.000 <score>=0.740540540541\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:56.958] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 166, \"duration\": 558, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:56 INFO 139813082523456] #quality_metric: host=algo-1, epoch=82, train binary_classification_accuracy <score>=0.746879120879\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:56 INFO 139813082523456] #quality_metric: host=algo-1, epoch=82, train binary_classification_cross_entropy <loss>=0.520813287756\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:56 INFO 139813082523456] #quality_metric: host=algo-1, epoch=82, train binary_f_1.000 <score>=0.776953616733\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 560.4739189147949, \"sum\": 560.4739189147949, \"min\": 560.4739189147949}}, \"EndTime\": 1604871656.959068, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871656.397964}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:56 INFO 139813082523456] #progress_metric: host=algo-1, completed 41 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 7554, \"sum\": 7554.0, \"min\": 7554}, \"Total Records Seen\": {\"count\": 1, \"max\": 7518310, \"sum\": 7518310.0, \"min\": 7518310}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 84, \"sum\": 84.0, \"min\": 84}}, \"EndTime\": 1604871656.959284, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 82}, \"StartTime\": 1604871656.398563}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:56 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=161498.521083 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:56 INFO 139813082523456] #quality_metric: host=algo-1, epoch=83, batch=0 train binary_classification_accuracy <score>=0.712\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:56 INFO 139813082523456] #quality_metric: host=algo-1, epoch=83, batch=0 train binary_classification_cross_entropy <loss>=0.5427109375\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:56 INFO 139813082523456] #quality_metric: host=algo-1, epoch=83, batch=0 train binary_f_1.000 <score>=0.740540540541\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:57.500] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 168, \"duration\": 539, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:57 INFO 139813082523456] #quality_metric: host=algo-1, epoch=83, train binary_classification_accuracy <score>=0.746901098901\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:57 INFO 139813082523456] #quality_metric: host=algo-1, epoch=83, train binary_classification_cross_entropy <loss>=0.520423303625\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:57 INFO 139813082523456] #quality_metric: host=algo-1, epoch=83, train binary_f_1.000 <score>=0.776985940586\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 540.8010482788086, \"sum\": 540.8010482788086, \"min\": 540.8010482788086}}, \"EndTime\": 1604871657.500564, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871656.959136}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:57 INFO 139813082523456] #progress_metric: host=algo-1, completed 42 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 7645, \"sum\": 7645.0, \"min\": 7645}, \"Total Records Seen\": {\"count\": 1, \"max\": 7608880, \"sum\": 7608880.0, \"min\": 7608880}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 85, \"sum\": 85.0, \"min\": 85}}, \"EndTime\": 1604871657.500768, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 83}, \"StartTime\": 1604871656.959733}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:57 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=167367.019827 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:57 INFO 139813082523456] #quality_metric: host=algo-1, epoch=84, batch=0 train binary_classification_accuracy <score>=0.711\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:57 INFO 139813082523456] #quality_metric: host=algo-1, epoch=84, batch=0 train binary_classification_cross_entropy <loss>=0.542393676758\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:57 INFO 139813082523456] #quality_metric: host=algo-1, epoch=84, batch=0 train binary_f_1.000 <score>=0.739873987399\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:58.066] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 170, \"duration\": 563, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:58 INFO 139813082523456] #quality_metric: host=algo-1, epoch=84, train binary_classification_accuracy <score>=0.747021978022\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:58 INFO 139813082523456] #quality_metric: host=algo-1, epoch=84, train binary_classification_cross_entropy <loss>=0.520038545043\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:58 INFO 139813082523456] #quality_metric: host=algo-1, epoch=84, train binary_f_1.000 <score>=0.7770773417\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 565.3109550476074, \"sum\": 565.3109550476074, \"min\": 565.3109550476074}}, \"EndTime\": 1604871658.066623, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871657.500633}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:58 INFO 139813082523456] #progress_metric: host=algo-1, completed 42 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 7736, \"sum\": 7736.0, \"min\": 7736}, \"Total Records Seen\": {\"count\": 1, \"max\": 7699450, \"sum\": 7699450.0, \"min\": 7699450}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 86, \"sum\": 86.0, \"min\": 86}}, \"EndTime\": 1604871658.06683, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 84}, \"StartTime\": 1604871657.501285}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:58 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=160117.831123 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:58 INFO 139813082523456] #quality_metric: host=algo-1, epoch=85, batch=0 train binary_classification_accuracy <score>=0.711\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:58 INFO 139813082523456] #quality_metric: host=algo-1, epoch=85, batch=0 train binary_classification_cross_entropy <loss>=0.542079528809\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:58 INFO 139813082523456] #quality_metric: host=algo-1, epoch=85, batch=0 train binary_f_1.000 <score>=0.739873987399\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[34m[2020-11-08 21:40:58.602] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 172, \"duration\": 533, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:58 INFO 139813082523456] #quality_metric: host=algo-1, epoch=85, train binary_classification_accuracy <score>=0.747032967033\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:58 INFO 139813082523456] #quality_metric: host=algo-1, epoch=85, train binary_classification_cross_entropy <loss>=0.519658784971\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:58 INFO 139813082523456] #quality_metric: host=algo-1, epoch=85, train binary_f_1.000 <score>=0.777093500658\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 535.3460311889648, \"sum\": 535.3460311889648, \"min\": 535.3460311889648}}, \"EndTime\": 1604871658.602676, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871658.066683}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:58 INFO 139813082523456] #progress_metric: host=algo-1, completed 43 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 7827, \"sum\": 7827.0, \"min\": 7827}, \"Total Records Seen\": {\"count\": 1, \"max\": 7790020, \"sum\": 7790020.0, \"min\": 7790020}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 87, \"sum\": 87.0, \"min\": 87}}, \"EndTime\": 1604871658.602825, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 85}, \"StartTime\": 1604871658.067304}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:58 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=169099.112871 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:58 INFO 139813082523456] #quality_metric: host=algo-1, epoch=86, batch=0 train binary_classification_accuracy <score>=0.711\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:58 INFO 139813082523456] #quality_metric: host=algo-1, epoch=86, batch=0 train binary_classification_cross_entropy <loss>=0.541768371582\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:58 INFO 139813082523456] #quality_metric: host=algo-1, epoch=86, batch=0 train binary_f_1.000 <score>=0.739873987399\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:59.147] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 174, \"duration\": 543, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:59 INFO 139813082523456] #quality_metric: host=algo-1, epoch=86, train binary_classification_accuracy <score>=0.74721978022\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:59 INFO 139813082523456] #quality_metric: host=algo-1, epoch=86, train binary_classification_cross_entropy <loss>=0.519283804422\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:59 INFO 139813082523456] #quality_metric: host=algo-1, epoch=86, train binary_f_1.000 <score>=0.777268898206\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 545.1061725616455, \"sum\": 545.1061725616455, \"min\": 545.1061725616455}}, \"EndTime\": 1604871659.148458, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871658.602732}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:59 INFO 139813082523456] #progress_metric: host=algo-1, completed 43 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 7918, \"sum\": 7918.0, \"min\": 7918}, \"Total Records Seen\": {\"count\": 1, \"max\": 7880590, \"sum\": 7880590.0, \"min\": 7880590}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 88, \"sum\": 88.0, \"min\": 88}}, \"EndTime\": 1604871659.148659, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 86}, \"StartTime\": 1604871658.603321}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:59 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=166050.021257 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:59 INFO 139813082523456] #quality_metric: host=algo-1, epoch=87, batch=0 train binary_classification_accuracy <score>=0.711\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:59 INFO 139813082523456] #quality_metric: host=algo-1, epoch=87, batch=0 train binary_classification_cross_entropy <loss>=0.541459960938\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:59 INFO 139813082523456] #quality_metric: host=algo-1, epoch=87, batch=0 train binary_f_1.000 <score>=0.739873987399\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:40:59.728] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 176, \"duration\": 578, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:59 INFO 139813082523456] #quality_metric: host=algo-1, epoch=87, train binary_classification_accuracy <score>=0.747395604396\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:59 INFO 139813082523456] #quality_metric: host=algo-1, epoch=87, train binary_classification_cross_entropy <loss>=0.518913389436\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:59 INFO 139813082523456] #quality_metric: host=algo-1, epoch=87, train binary_f_1.000 <score>=0.777402267907\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 580.1889896392822, \"sum\": 580.1889896392822, \"min\": 580.1889896392822}}, \"EndTime\": 1604871659.729354, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871659.148517}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:59 INFO 139813082523456] #progress_metric: host=algo-1, completed 44 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 8009, \"sum\": 8009.0, \"min\": 8009}, \"Total Records Seen\": {\"count\": 1, \"max\": 7971160, \"sum\": 7971160.0, \"min\": 7971160}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 89, \"sum\": 89.0, \"min\": 89}}, \"EndTime\": 1604871659.729504, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 87}, \"StartTime\": 1604871659.14914}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:59 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=156033.583155 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:59 INFO 139813082523456] #quality_metric: host=algo-1, epoch=88, batch=0 train binary_classification_accuracy <score>=0.711\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:59 INFO 139813082523456] #quality_metric: host=algo-1, epoch=88, batch=0 train binary_classification_cross_entropy <loss>=0.541154174805\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:40:59 INFO 139813082523456] #quality_metric: host=algo-1, epoch=88, batch=0 train binary_f_1.000 <score>=0.739873987399\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:00.304] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 178, \"duration\": 573, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:00 INFO 139813082523456] #quality_metric: host=algo-1, epoch=88, train binary_classification_accuracy <score>=0.747626373626\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:00 INFO 139813082523456] #quality_metric: host=algo-1, epoch=88, train binary_classification_cross_entropy <loss>=0.518547332093\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:00 INFO 139813082523456] #quality_metric: host=algo-1, epoch=88, train binary_f_1.000 <score>=0.777594855804\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 575.0000476837158, \"sum\": 575.0000476837158, \"min\": 575.0000476837158}}, \"EndTime\": 1604871660.305021, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871659.729411}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:00 INFO 139813082523456] #progress_metric: host=algo-1, completed 44 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 8100, \"sum\": 8100.0, \"min\": 8100}, \"Total Records Seen\": {\"count\": 1, \"max\": 8061730, \"sum\": 8061730.0, \"min\": 8061730}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 90, \"sum\": 90.0, \"min\": 90}}, \"EndTime\": 1604871660.305207, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 88}, \"StartTime\": 1604871659.729993}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:00 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=157424.909412 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:00 INFO 139813082523456] #quality_metric: host=algo-1, epoch=89, batch=0 train binary_classification_accuracy <score>=0.712\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:00 INFO 139813082523456] #quality_metric: host=algo-1, epoch=89, batch=0 train binary_classification_cross_entropy <loss>=0.540850830078\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:00 INFO 139813082523456] #quality_metric: host=algo-1, epoch=89, batch=0 train binary_f_1.000 <score>=0.741007194245\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:00.851] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 180, \"duration\": 544, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:00 INFO 139813082523456] #quality_metric: host=algo-1, epoch=89, train binary_classification_accuracy <score>=0.747879120879\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:00 INFO 139813082523456] #quality_metric: host=algo-1, epoch=89, train binary_classification_cross_entropy <loss>=0.518185426146\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:00 INFO 139813082523456] #quality_metric: host=algo-1, epoch=89, train binary_f_1.000 <score>=0.777806831498\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 545.706033706665, \"sum\": 545.706033706665, \"min\": 545.706033706665}}, \"EndTime\": 1604871660.851434, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871660.305081}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:00 INFO 139813082523456] #progress_metric: host=algo-1, completed 45 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 8191, \"sum\": 8191.0, \"min\": 8191}, \"Total Records Seen\": {\"count\": 1, \"max\": 8152300, \"sum\": 8152300.0, \"min\": 8152300}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}}, \"EndTime\": 1604871660.851582, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 89}, \"StartTime\": 1604871660.305701}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:00 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=165889.768121 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:00 INFO 139813082523456] #quality_metric: host=algo-1, epoch=90, batch=0 train binary_classification_accuracy <score>=0.713\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:00 INFO 139813082523456] #quality_metric: host=algo-1, epoch=90, batch=0 train binary_classification_cross_entropy <loss>=0.540549682617\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:00 INFO 139813082523456] #quality_metric: host=algo-1, epoch=90, batch=0 train binary_f_1.000 <score>=0.741674167417\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:01.403] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 182, \"duration\": 550, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:01 INFO 139813082523456] #quality_metric: host=algo-1, epoch=90, train binary_classification_accuracy <score>=0.748054945055\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:01 INFO 139813082523456] #quality_metric: host=algo-1, epoch=90, train binary_classification_cross_entropy <loss>=0.517827479771\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:01 INFO 139813082523456] #quality_metric: host=algo-1, epoch=90, train binary_f_1.000 <score>=0.777931675755\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 552.1931648254395, \"sum\": 552.1931648254395, \"min\": 552.1931648254395}}, \"EndTime\": 1604871661.404279, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871660.85149}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:01 INFO 139813082523456] #progress_metric: host=algo-1, completed 45 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 8282, \"sum\": 8282.0, \"min\": 8282}, \"Total Records Seen\": {\"count\": 1, \"max\": 8242870, \"sum\": 8242870.0, \"min\": 8242870}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 92, \"sum\": 92.0, \"min\": 92}}, \"EndTime\": 1604871661.404467, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 90}, \"StartTime\": 1604871660.852062}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:01 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=163924.62799 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:01 INFO 139813082523456] #quality_metric: host=algo-1, epoch=91, batch=0 train binary_classification_accuracy <score>=0.715\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:01 INFO 139813082523456] #quality_metric: host=algo-1, epoch=91, batch=0 train binary_classification_cross_entropy <loss>=0.540250671387\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:01 INFO 139813082523456] #quality_metric: host=algo-1, epoch=91, batch=0 train binary_f_1.000 <score>=0.743011722272\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:02.009] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 184, \"duration\": 603, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:02 INFO 139813082523456] #quality_metric: host=algo-1, epoch=91, train binary_classification_accuracy <score>=0.748241758242\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:02 INFO 139813082523456] #quality_metric: host=algo-1, epoch=91, train binary_classification_cross_entropy <loss>=0.517473299131\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:02 INFO 139813082523456] #quality_metric: host=algo-1, epoch=91, train binary_f_1.000 <score>=0.778081289473\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 605.0889492034912, \"sum\": 605.0889492034912, \"min\": 605.0889492034912}}, \"EndTime\": 1604871662.010065, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871661.404337}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:02 INFO 139813082523456] #progress_metric: host=algo-1, completed 46 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 8373, \"sum\": 8373.0, \"min\": 8373}, \"Total Records Seen\": {\"count\": 1, \"max\": 8333440, \"sum\": 8333440.0, \"min\": 8333440}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 93, \"sum\": 93.0, \"min\": 93}}, \"EndTime\": 1604871662.010246, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 91}, \"StartTime\": 1604871661.404948}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:02 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=149604.312696 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:02 INFO 139813082523456] #quality_metric: host=algo-1, epoch=92, batch=0 train binary_classification_accuracy <score>=0.716\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:02 INFO 139813082523456] #quality_metric: host=algo-1, epoch=92, batch=0 train binary_classification_cross_entropy <loss>=0.539953491211\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:02 INFO 139813082523456] #quality_metric: host=algo-1, epoch=92, batch=0 train binary_f_1.000 <score>=0.744144144144\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:02.572] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 186, \"duration\": 561, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:02 INFO 139813082523456] #quality_metric: host=algo-1, epoch=92, train binary_classification_accuracy <score>=0.74843956044\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:02 INFO 139813082523456] #quality_metric: host=algo-1, epoch=92, train binary_classification_cross_entropy <loss>=0.517122704474\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:02 INFO 139813082523456] #quality_metric: host=algo-1, epoch=92, train binary_f_1.000 <score>=0.778234165811\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 562.6668930053711, \"sum\": 562.6668930053711, \"min\": 562.6668930053711}}, \"EndTime\": 1604871662.573434, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871662.010124}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:02 INFO 139813082523456] #progress_metric: host=algo-1, completed 46 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 8464, \"sum\": 8464.0, \"min\": 8464}, \"Total Records Seen\": {\"count\": 1, \"max\": 8424010, \"sum\": 8424010.0, \"min\": 8424010}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 94, \"sum\": 94.0, \"min\": 94}}, \"EndTime\": 1604871662.573629, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 92}, \"StartTime\": 1604871662.010738}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:02 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=160876.070552 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:02 INFO 139813082523456] #quality_metric: host=algo-1, epoch=93, batch=0 train binary_classification_accuracy <score>=0.716\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:02 INFO 139813082523456] #quality_metric: host=algo-1, epoch=93, batch=0 train binary_classification_cross_entropy <loss>=0.53965802002\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:02 INFO 139813082523456] #quality_metric: host=algo-1, epoch=93, batch=0 train binary_f_1.000 <score>=0.744144144144\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:03.134] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 188, \"duration\": 559, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:03 INFO 139813082523456] #quality_metric: host=algo-1, epoch=93, train binary_classification_accuracy <score>=0.748692307692\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:03 INFO 139813082523456] #quality_metric: host=algo-1, epoch=93, train binary_classification_cross_entropy <loss>=0.516775512025\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:03 INFO 139813082523456] #quality_metric: host=algo-1, epoch=93, train binary_f_1.000 <score>=0.778480583512\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 561.129093170166, \"sum\": 561.129093170166, \"min\": 561.129093170166}}, \"EndTime\": 1604871663.135278, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871662.573488}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:03 INFO 139813082523456] #progress_metric: host=algo-1, completed 47 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 8555, \"sum\": 8555.0, \"min\": 8555}, \"Total Records Seen\": {\"count\": 1, \"max\": 8514580, \"sum\": 8514580.0, \"min\": 8514580}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 95, \"sum\": 95.0, \"min\": 95}}, \"EndTime\": 1604871663.135469, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 93}, \"StartTime\": 1604871662.574119}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:03 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=161317.055444 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:03 INFO 139813082523456] #quality_metric: host=algo-1, epoch=94, batch=0 train binary_classification_accuracy <score>=0.716\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:03 INFO 139813082523456] #quality_metric: host=algo-1, epoch=94, batch=0 train binary_classification_cross_entropy <loss>=0.539364257812\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:03 INFO 139813082523456] #quality_metric: host=algo-1, epoch=94, batch=0 train binary_f_1.000 <score>=0.744144144144\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:03.688] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 190, \"duration\": 551, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:03 INFO 139813082523456] #quality_metric: host=algo-1, epoch=94, train binary_classification_accuracy <score>=0.748912087912\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:03 INFO 139813082523456] #quality_metric: host=algo-1, epoch=94, train binary_classification_cross_entropy <loss>=0.516431561481\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:03 INFO 139813082523456] #quality_metric: host=algo-1, epoch=94, train binary_f_1.000 <score>=0.778674312504\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 553.1370639801025, \"sum\": 553.1370639801025, \"min\": 553.1370639801025}}, \"EndTime\": 1604871663.689108, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871663.135334}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:03 INFO 139813082523456] #progress_metric: host=algo-1, completed 47 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 8646, \"sum\": 8646.0, \"min\": 8646}, \"Total Records Seen\": {\"count\": 1, \"max\": 8605150, \"sum\": 8605150.0, \"min\": 8605150}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 96, \"sum\": 96.0, \"min\": 96}}, \"EndTime\": 1604871663.689354, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 94}, \"StartTime\": 1604871663.135941}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:03 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=163626.098329 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:03 INFO 139813082523456] #quality_metric: host=algo-1, epoch=95, batch=0 train binary_classification_accuracy <score>=0.717\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:03 INFO 139813082523456] #quality_metric: host=algo-1, epoch=95, batch=0 train binary_classification_cross_entropy <loss>=0.539071899414\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:03 INFO 139813082523456] #quality_metric: host=algo-1, epoch=95, batch=0 train binary_f_1.000 <score>=0.744815148783\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:04.252] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 192, \"duration\": 560, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:04 INFO 139813082523456] #quality_metric: host=algo-1, epoch=95, train binary_classification_accuracy <score>=0.748978021978\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:04 INFO 139813082523456] #quality_metric: host=algo-1, epoch=95, train binary_classification_cross_entropy <loss>=0.516090674096\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:04 INFO 139813082523456] #quality_metric: host=algo-1, epoch=95, train binary_f_1.000 <score>=0.778715283496\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 562.5438690185547, \"sum\": 562.5438690185547, \"min\": 562.5438690185547}}, \"EndTime\": 1604871664.252413, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871663.689191}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:04 INFO 139813082523456] #progress_metric: host=algo-1, completed 48 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 8737, \"sum\": 8737.0, \"min\": 8737}, \"Total Records Seen\": {\"count\": 1, \"max\": 8695720, \"sum\": 8695720.0, \"min\": 8695720}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 97, \"sum\": 97.0, \"min\": 97}}, \"EndTime\": 1604871664.252564, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 95}, \"StartTime\": 1604871663.689844}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:04 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=160926.025377 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:04 INFO 139813082523456] #quality_metric: host=algo-1, epoch=96, batch=0 train binary_classification_accuracy <score>=0.717\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:04 INFO 139813082523456] #quality_metric: host=algo-1, epoch=96, batch=0 train binary_classification_cross_entropy <loss>=0.538780761719\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:04 INFO 139813082523456] #quality_metric: host=algo-1, epoch=96, batch=0 train binary_f_1.000 <score>=0.744815148783\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:04.821] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 194, \"duration\": 567, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:04 INFO 139813082523456] #quality_metric: host=algo-1, epoch=96, train binary_classification_accuracy <score>=0.749098901099\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:04 INFO 139813082523456] #quality_metric: host=algo-1, epoch=96, train binary_classification_cross_entropy <loss>=0.5157526889\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:04 INFO 139813082523456] #quality_metric: host=algo-1, epoch=96, train binary_f_1.000 <score>=0.778836839862\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 569.3778991699219, \"sum\": 569.3778991699219, \"min\": 569.3778991699219}}, \"EndTime\": 1604871664.822438, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871664.252469}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:04 INFO 139813082523456] #progress_metric: host=algo-1, completed 48 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 8828, \"sum\": 8828.0, \"min\": 8828}, \"Total Records Seen\": {\"count\": 1, \"max\": 8786290, \"sum\": 8786290.0, \"min\": 8786290}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 98, \"sum\": 98.0, \"min\": 98}}, \"EndTime\": 1604871664.822624, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 96}, \"StartTime\": 1604871664.253033}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:04 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=158983.7317 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:04 INFO 139813082523456] #quality_metric: host=algo-1, epoch=97, batch=0 train binary_classification_accuracy <score>=0.718\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:04 INFO 139813082523456] #quality_metric: host=algo-1, epoch=97, batch=0 train binary_classification_cross_entropy <loss>=0.538490966797\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:04 INFO 139813082523456] #quality_metric: host=algo-1, epoch=97, batch=0 train binary_f_1.000 <score>=0.745945945946\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:05.429] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 196, \"duration\": 605, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:05 INFO 139813082523456] #quality_metric: host=algo-1, epoch=97, train binary_classification_accuracy <score>=0.74921978022\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:05 INFO 139813082523456] #quality_metric: host=algo-1, epoch=97, train binary_classification_cross_entropy <loss>=0.51541745867\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:05 INFO 139813082523456] #quality_metric: host=algo-1, epoch=97, train binary_f_1.000 <score>=0.778941250545\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 607.1751117706299, \"sum\": 607.1751117706299, \"min\": 607.1751117706299}}, \"EndTime\": 1604871665.430341, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871664.822493}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:05 INFO 139813082523456] #progress_metric: host=algo-1, completed 49 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 8919, \"sum\": 8919.0, \"min\": 8919}, \"Total Records Seen\": {\"count\": 1, \"max\": 8876860, \"sum\": 8876860.0, \"min\": 8876860}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 99, \"sum\": 99.0, \"min\": 99}}, \"EndTime\": 1604871665.430535, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 97}, \"StartTime\": 1604871664.823138}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:05 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=149089.092549 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:05 INFO 139813082523456] #quality_metric: host=algo-1, epoch=98, batch=0 train binary_classification_accuracy <score>=0.719\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:05 INFO 139813082523456] #quality_metric: host=algo-1, epoch=98, batch=0 train binary_classification_cross_entropy <loss>=0.538202148438\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:05 INFO 139813082523456] #quality_metric: host=algo-1, epoch=98, batch=0 train binary_f_1.000 <score>=0.746618575293\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:06.024] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 198, \"duration\": 592, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:06 INFO 139813082523456] #quality_metric: host=algo-1, epoch=98, train binary_classification_accuracy <score>=0.749417582418\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:06 INFO 139813082523456] #quality_metric: host=algo-1, epoch=98, train binary_classification_cross_entropy <loss>=0.515084816734\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:06 INFO 139813082523456] #quality_metric: host=algo-1, epoch=98, train binary_f_1.000 <score>=0.779124168192\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 593.9538478851318, \"sum\": 593.9538478851318, \"min\": 593.9538478851318}}, \"EndTime\": 1604871666.024957, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871665.430402}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:06 INFO 139813082523456] #progress_metric: host=algo-1, completed 49 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 9010, \"sum\": 9010.0, \"min\": 9010}, \"Total Records Seen\": {\"count\": 1, \"max\": 8967430, \"sum\": 8967430.0, \"min\": 8967430}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 100, \"sum\": 100.0, \"min\": 100}}, \"EndTime\": 1604871666.025131, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 98}, \"StartTime\": 1604871665.430977}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:06 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=152411.650426 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:06 INFO 139813082523456] #quality_metric: host=algo-1, epoch=99, batch=0 train binary_classification_accuracy <score>=0.72\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:06 INFO 139813082523456] #quality_metric: host=algo-1, epoch=99, batch=0 train binary_classification_cross_entropy <loss>=0.53791418457\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:06 INFO 139813082523456] #quality_metric: host=algo-1, epoch=99, batch=0 train binary_f_1.000 <score>=0.747747747748\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:06.585] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 200, \"duration\": 558, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:06 INFO 139813082523456] #quality_metric: host=algo-1, epoch=99, train binary_classification_accuracy <score>=0.749747252747\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:06 INFO 139813082523456] #quality_metric: host=algo-1, epoch=99, train binary_classification_cross_entropy <loss>=0.514754623245\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:06 INFO 139813082523456] #quality_metric: host=algo-1, epoch=99, train binary_f_1.000 <score>=0.779397661555\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 560.4867935180664, \"sum\": 560.4867935180664, \"min\": 560.4867935180664}}, \"EndTime\": 1604871666.586115, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871666.025014}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:06 INFO 139813082523456] #progress_metric: host=algo-1, completed 50 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 9101, \"sum\": 9101.0, \"min\": 9101}, \"Total Records Seen\": {\"count\": 1, \"max\": 9058000, \"sum\": 9058000.0, \"min\": 9058000}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 101, \"sum\": 101.0, \"min\": 101}}, \"EndTime\": 1604871666.586305, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 99}, \"StartTime\": 1604871666.0256}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:06 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=161501.748086 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:06 INFO 139813082523456] #quality_metric: host=algo-1, epoch=100, batch=0 train binary_classification_accuracy <score>=0.72\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:06 INFO 139813082523456] #quality_metric: host=algo-1, epoch=100, batch=0 train binary_classification_cross_entropy <loss>=0.53762701416\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:06 INFO 139813082523456] #quality_metric: host=algo-1, epoch=100, batch=0 train binary_f_1.000 <score>=0.747747747748\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:07.131] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 202, \"duration\": 543, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:07 INFO 139813082523456] #quality_metric: host=algo-1, epoch=100, train binary_classification_accuracy <score>=0.749813186813\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:07 INFO 139813082523456] #quality_metric: host=algo-1, epoch=100, train binary_classification_cross_entropy <loss>=0.514426732325\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:07 INFO 139813082523456] #quality_metric: host=algo-1, epoch=100, train binary_f_1.000 <score>=0.779451510719\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 545.3948974609375, \"sum\": 545.3948974609375, \"min\": 545.3948974609375}}, \"EndTime\": 1604871667.132226, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871666.586168}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:07 INFO 139813082523456] #progress_metric: host=algo-1, completed 50 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 9192, \"sum\": 9192.0, \"min\": 9192}, \"Total Records Seen\": {\"count\": 1, \"max\": 9148570, \"sum\": 9148570.0, \"min\": 9148570}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 102, \"sum\": 102.0, \"min\": 102}}, \"EndTime\": 1604871667.132379, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 100}, \"StartTime\": 1604871666.586805}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:07 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=165982.474177 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:07 INFO 139813082523456] #quality_metric: host=algo-1, epoch=101, batch=0 train binary_classification_accuracy <score>=0.719\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:07 INFO 139813082523456] #quality_metric: host=algo-1, epoch=101, batch=0 train binary_classification_cross_entropy <loss>=0.537340576172\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:07 INFO 139813082523456] #quality_metric: host=algo-1, epoch=101, batch=0 train binary_f_1.000 <score>=0.747074707471\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:07.689] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 204, \"duration\": 555, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:07 INFO 139813082523456] #quality_metric: host=algo-1, epoch=101, train binary_classification_accuracy <score>=0.749901098901\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:07 INFO 139813082523456] #quality_metric: host=algo-1, epoch=101, train binary_classification_cross_entropy <loss>=0.514101010501\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:07 INFO 139813082523456] #quality_metric: host=algo-1, epoch=101, train binary_f_1.000 <score>=0.779524736745\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 557.142972946167, \"sum\": 557.142972946167, \"min\": 557.142972946167}}, \"EndTime\": 1604871667.690035, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871667.132282}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:07 INFO 139813082523456] #progress_metric: host=algo-1, completed 51 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 9283, \"sum\": 9283.0, \"min\": 9283}, \"Total Records Seen\": {\"count\": 1, \"max\": 9239140, \"sum\": 9239140.0, \"min\": 9239140}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 103, \"sum\": 103.0, \"min\": 103}}, \"EndTime\": 1604871667.690192, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 101}, \"StartTime\": 1604871667.132853}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:07 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=162480.432268 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:07 INFO 139813082523456] #quality_metric: host=algo-1, epoch=102, batch=0 train binary_classification_accuracy <score>=0.719\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:07 INFO 139813082523456] #quality_metric: host=algo-1, epoch=102, batch=0 train binary_classification_cross_entropy <loss>=0.53705456543\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:07 INFO 139813082523456] #quality_metric: host=algo-1, epoch=102, batch=0 train binary_f_1.000 <score>=0.747074707471\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[34m[2020-11-08 21:41:08.238] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 206, \"duration\": 547, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:08 INFO 139813082523456] #quality_metric: host=algo-1, epoch=102, train binary_classification_accuracy <score>=0.749967032967\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:08 INFO 139813082523456] #quality_metric: host=algo-1, epoch=102, train binary_classification_cross_entropy <loss>=0.513777320275\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:08 INFO 139813082523456] #quality_metric: host=algo-1, epoch=102, train binary_f_1.000 <score>=0.779587131523\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 548.6500263214111, \"sum\": 548.6500263214111, \"min\": 548.6500263214111}}, \"EndTime\": 1604871668.239358, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871667.690092}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:08 INFO 139813082523456] #progress_metric: host=algo-1, completed 51 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 9374, \"sum\": 9374.0, \"min\": 9374}, \"Total Records Seen\": {\"count\": 1, \"max\": 9329710, \"sum\": 9329710.0, \"min\": 9329710}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 104, \"sum\": 104.0, \"min\": 104}}, \"EndTime\": 1604871668.239507, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 102}, \"StartTime\": 1604871667.690681}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:08 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=164999.469348 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:08 INFO 139813082523456] #quality_metric: host=algo-1, epoch=103, batch=0 train binary_classification_accuracy <score>=0.72\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:08 INFO 139813082523456] #quality_metric: host=algo-1, epoch=103, batch=0 train binary_classification_cross_entropy <loss>=0.536769042969\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:08 INFO 139813082523456] #quality_metric: host=algo-1, epoch=103, batch=0 train binary_f_1.000 <score>=0.747747747748\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:08.789] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 208, \"duration\": 547, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:08 INFO 139813082523456] #quality_metric: host=algo-1, epoch=103, train binary_classification_accuracy <score>=0.750032967033\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:08 INFO 139813082523456] #quality_metric: host=algo-1, epoch=103, train binary_classification_cross_entropy <loss>=0.513455534547\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:08 INFO 139813082523456] #quality_metric: host=algo-1, epoch=103, train binary_f_1.000 <score>=0.779632446257\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 549.4329929351807, \"sum\": 549.4329929351807, \"min\": 549.4329929351807}}, \"EndTime\": 1604871668.789461, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871668.239414}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:08 INFO 139813082523456] #progress_metric: host=algo-1, completed 52 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 9465, \"sum\": 9465.0, \"min\": 9465}, \"Total Records Seen\": {\"count\": 1, \"max\": 9420280, \"sum\": 9420280.0, \"min\": 9420280}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 105, \"sum\": 105.0, \"min\": 105}}, \"EndTime\": 1604871668.789613, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 103}, \"StartTime\": 1604871668.240001}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:08 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=164763.52027 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:08 INFO 139813082523456] #quality_metric: host=algo-1, epoch=104, batch=0 train binary_classification_accuracy <score>=0.72\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:08 INFO 139813082523456] #quality_metric: host=algo-1, epoch=104, batch=0 train binary_classification_cross_entropy <loss>=0.536483764648\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:08 INFO 139813082523456] #quality_metric: host=algo-1, epoch=104, batch=0 train binary_f_1.000 <score>=0.747747747748\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:09.343] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 210, \"duration\": 552, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:09 INFO 139813082523456] #quality_metric: host=algo-1, epoch=104, train binary_classification_accuracy <score>=0.750065934066\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:09 INFO 139813082523456] #quality_metric: host=algo-1, epoch=104, train binary_classification_cross_entropy <loss>=0.513135523199\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:09 INFO 139813082523456] #quality_metric: host=algo-1, epoch=104, train binary_f_1.000 <score>=0.779659374939\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 553.962230682373, \"sum\": 553.962230682373, \"min\": 553.962230682373}}, \"EndTime\": 1604871669.344132, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871668.789518}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:09 INFO 139813082523456] #progress_metric: host=algo-1, completed 52 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 9556, \"sum\": 9556.0, \"min\": 9556}, \"Total Records Seen\": {\"count\": 1, \"max\": 9510850, \"sum\": 9510850.0, \"min\": 9510850}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 106, \"sum\": 106.0, \"min\": 106}}, \"EndTime\": 1604871669.344281, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 104}, \"StartTime\": 1604871668.790144}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:09 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=163417.394084 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:09 INFO 139813082523456] #quality_metric: host=algo-1, epoch=105, batch=0 train binary_classification_accuracy <score>=0.721\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:09 INFO 139813082523456] #quality_metric: host=algo-1, epoch=105, batch=0 train binary_classification_cross_entropy <loss>=0.536198791504\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:09 INFO 139813082523456] #quality_metric: host=algo-1, epoch=105, batch=0 train binary_f_1.000 <score>=0.748422001803\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:09.913] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 212, \"duration\": 567, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:09 INFO 139813082523456] #quality_metric: host=algo-1, epoch=105, train binary_classification_accuracy <score>=0.750252747253\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:09 INFO 139813082523456] #quality_metric: host=algo-1, epoch=105, train binary_classification_cross_entropy <loss>=0.512817176903\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:09 INFO 139813082523456] #quality_metric: host=algo-1, epoch=105, train binary_f_1.000 <score>=0.779838998731\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 568.6450004577637, \"sum\": 568.6450004577637, \"min\": 568.6450004577637}}, \"EndTime\": 1604871669.913434, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871669.344184}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:09 INFO 139813082523456] #progress_metric: host=algo-1, completed 53 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 9647, \"sum\": 9647.0, \"min\": 9647}, \"Total Records Seen\": {\"count\": 1, \"max\": 9601420, \"sum\": 9601420.0, \"min\": 9601420}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 107, \"sum\": 107.0, \"min\": 107}}, \"EndTime\": 1604871669.913577, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 105}, \"StartTime\": 1604871669.344763}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:09 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=159202.538872 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:09 INFO 139813082523456] #quality_metric: host=algo-1, epoch=106, batch=0 train binary_classification_accuracy <score>=0.721\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:09 INFO 139813082523456] #quality_metric: host=algo-1, epoch=106, batch=0 train binary_classification_cross_entropy <loss>=0.53591394043\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:09 INFO 139813082523456] #quality_metric: host=algo-1, epoch=106, batch=0 train binary_f_1.000 <score>=0.748422001803\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:10.474] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 214, \"duration\": 559, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:10 INFO 139813082523456] #quality_metric: host=algo-1, epoch=106, train binary_classification_accuracy <score>=0.75032967033\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:10 INFO 139813082523456] #quality_metric: host=algo-1, epoch=106, train binary_classification_cross_entropy <loss>=0.512500353803\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:10 INFO 139813082523456] #quality_metric: host=algo-1, epoch=106, train binary_f_1.000 <score>=0.779925996242\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 561.089038848877, \"sum\": 561.089038848877, \"min\": 561.089038848877}}, \"EndTime\": 1604871670.475208, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871669.913488}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:10 INFO 139813082523456] #progress_metric: host=algo-1, completed 53 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 9738, \"sum\": 9738.0, \"min\": 9738}, \"Total Records Seen\": {\"count\": 1, \"max\": 9691990, \"sum\": 9691990.0, \"min\": 9691990}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 108, \"sum\": 108.0, \"min\": 108}}, \"EndTime\": 1604871670.475361, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 106}, \"StartTime\": 1604871669.914094}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:10 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=161332.538847 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:10 INFO 139813082523456] #quality_metric: host=algo-1, epoch=107, batch=0 train binary_classification_accuracy <score>=0.721\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:10 INFO 139813082523456] #quality_metric: host=algo-1, epoch=107, batch=0 train binary_classification_cross_entropy <loss>=0.53562902832\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:10 INFO 139813082523456] #quality_metric: host=algo-1, epoch=107, batch=0 train binary_f_1.000 <score>=0.748422001803\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:11.022] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 216, \"duration\": 545, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:11 INFO 139813082523456] #quality_metric: host=algo-1, epoch=107, train binary_classification_accuracy <score>=0.750538461538\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:11 INFO 139813082523456] #quality_metric: host=algo-1, epoch=107, train binary_classification_cross_entropy <loss>=0.512184959998\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:11 INFO 139813082523456] #quality_metric: host=algo-1, epoch=107, train binary_f_1.000 <score>=0.780116426613\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 547.1148490905762, \"sum\": 547.1148490905762, \"min\": 547.1148490905762}}, \"EndTime\": 1604871671.022996, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871670.475264}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:11 INFO 139813082523456] #progress_metric: host=algo-1, completed 54 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 9829, \"sum\": 9829.0, \"min\": 9829}, \"Total Records Seen\": {\"count\": 1, \"max\": 9782560, \"sum\": 9782560.0, \"min\": 9782560}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 109, \"sum\": 109.0, \"min\": 109}}, \"EndTime\": 1604871671.023142, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 107}, \"StartTime\": 1604871670.475854}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:11 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=165462.733581 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:11 INFO 139813082523456] #quality_metric: host=algo-1, epoch=108, batch=0 train binary_classification_accuracy <score>=0.721\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:11 INFO 139813082523456] #quality_metric: host=algo-1, epoch=108, batch=0 train binary_classification_cross_entropy <loss>=0.535343994141\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:11 INFO 139813082523456] #quality_metric: host=algo-1, epoch=108, batch=0 train binary_f_1.000 <score>=0.748422001803\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:11.574] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 218, \"duration\": 549, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:11 INFO 139813082523456] #quality_metric: host=algo-1, epoch=108, train binary_classification_accuracy <score>=0.750626373626\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:11 INFO 139813082523456] #quality_metric: host=algo-1, epoch=108, train binary_classification_cross_entropy <loss>=0.511870884487\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:11 INFO 139813082523456] #quality_metric: host=algo-1, epoch=108, train binary_f_1.000 <score>=0.780215203727\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 551.2099266052246, \"sum\": 551.2099266052246, \"min\": 551.2099266052246}}, \"EndTime\": 1604871671.574878, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871671.02305}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:11 INFO 139813082523456] #progress_metric: host=algo-1, completed 54 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 9920, \"sum\": 9920.0, \"min\": 9920}, \"Total Records Seen\": {\"count\": 1, \"max\": 9873130, \"sum\": 9873130.0, \"min\": 9873130}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 110, \"sum\": 110.0, \"min\": 110}}, \"EndTime\": 1604871671.575098, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 108}, \"StartTime\": 1604871671.023638}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:11 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=164202.527231 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:11 INFO 139813082523456] #quality_metric: host=algo-1, epoch=109, batch=0 train binary_classification_accuracy <score>=0.722\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:11 INFO 139813082523456] #quality_metric: host=algo-1, epoch=109, batch=0 train binary_classification_cross_entropy <loss>=0.535058837891\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:11 INFO 139813082523456] #quality_metric: host=algo-1, epoch=109, batch=0 train binary_f_1.000 <score>=0.749097472924\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:12.160] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 220, \"duration\": 583, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:12 INFO 139813082523456] #quality_metric: host=algo-1, epoch=109, train binary_classification_accuracy <score>=0.750703296703\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:12 INFO 139813082523456] #quality_metric: host=algo-1, epoch=109, train binary_classification_cross_entropy <loss>=0.511558010562\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:12 INFO 139813082523456] #quality_metric: host=algo-1, epoch=109, train binary_f_1.000 <score>=0.780306405067\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 585.3540897369385, \"sum\": 585.3540897369385, \"min\": 585.3540897369385}}, \"EndTime\": 1604871672.161001, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871671.574945}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:12 INFO 139813082523456] #progress_metric: host=algo-1, completed 55 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 10011, \"sum\": 10011.0, \"min\": 10011}, \"Total Records Seen\": {\"count\": 1, \"max\": 9963700, \"sum\": 9963700.0, \"min\": 9963700}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 111, \"sum\": 111.0, \"min\": 111}}, \"EndTime\": 1604871672.161196, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 109}, \"StartTime\": 1604871671.575618}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:12 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=154641.323926 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:12 INFO 139813082523456] #quality_metric: host=algo-1, epoch=110, batch=0 train binary_classification_accuracy <score>=0.721\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:12 INFO 139813082523456] #quality_metric: host=algo-1, epoch=110, batch=0 train binary_classification_cross_entropy <loss>=0.5347734375\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:12 INFO 139813082523456] #quality_metric: host=algo-1, epoch=110, batch=0 train binary_f_1.000 <score>=0.748422001803\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:12.725] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 222, \"duration\": 562, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:12 INFO 139813082523456] #quality_metric: host=algo-1, epoch=110, train binary_classification_accuracy <score>=0.750802197802\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:12 INFO 139813082523456] #quality_metric: host=algo-1, epoch=110, train binary_classification_cross_entropy <loss>=0.511246240972\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:12 INFO 139813082523456] #quality_metric: host=algo-1, epoch=110, train binary_f_1.000 <score>=0.780404194951\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 564.1989707946777, \"sum\": 564.1989707946777, \"min\": 564.1989707946777}}, \"EndTime\": 1604871672.725925, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871672.161056}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:12 INFO 139813082523456] #progress_metric: host=algo-1, completed 55 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 10102, \"sum\": 10102.0, \"min\": 10102}, \"Total Records Seen\": {\"count\": 1, \"max\": 10054270, \"sum\": 10054270.0, \"min\": 10054270}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 112, \"sum\": 112.0, \"min\": 112}}, \"EndTime\": 1604871672.726157, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 110}, \"StartTime\": 1604871672.161699}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:12 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=160424.413653 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:12 INFO 139813082523456] #quality_metric: host=algo-1, epoch=111, batch=0 train binary_classification_accuracy <score>=0.72\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:12 INFO 139813082523456] #quality_metric: host=algo-1, epoch=111, batch=0 train binary_classification_cross_entropy <loss>=0.534487670898\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:12 INFO 139813082523456] #quality_metric: host=algo-1, epoch=111, batch=0 train binary_f_1.000 <score>=0.747747747748\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:13.280] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 224, \"duration\": 552, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:13 INFO 139813082523456] #quality_metric: host=algo-1, epoch=111, train binary_classification_accuracy <score>=0.750879120879\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:13 INFO 139813082523456] #quality_metric: host=algo-1, epoch=111, train binary_classification_cross_entropy <loss>=0.510935478797\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:13 INFO 139813082523456] #quality_metric: host=algo-1, epoch=111, train binary_f_1.000 <score>=0.780482609032\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 553.9751052856445, \"sum\": 553.9751052856445, \"min\": 553.9751052856445}}, \"EndTime\": 1604871673.280672, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871672.725987}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:13 INFO 139813082523456] #progress_metric: host=algo-1, completed 56 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 10193, \"sum\": 10193.0, \"min\": 10193}, \"Total Records Seen\": {\"count\": 1, \"max\": 10144840, \"sum\": 10144840.0, \"min\": 10144840}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 113, \"sum\": 113.0, \"min\": 113}}, \"EndTime\": 1604871673.280819, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 111}, \"StartTime\": 1604871672.726672}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:13 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=163414.722747 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:13 INFO 139813082523456] #quality_metric: host=algo-1, epoch=112, batch=0 train binary_classification_accuracy <score>=0.72\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:13 INFO 139813082523456] #quality_metric: host=algo-1, epoch=112, batch=0 train binary_classification_cross_entropy <loss>=0.534201416016\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:13 INFO 139813082523456] #quality_metric: host=algo-1, epoch=112, batch=0 train binary_f_1.000 <score>=0.747747747748\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:13.824] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 226, \"duration\": 542, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:13 INFO 139813082523456] #quality_metric: host=algo-1, epoch=112, train binary_classification_accuracy <score>=0.751021978022\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:13 INFO 139813082523456] #quality_metric: host=algo-1, epoch=112, train binary_classification_cross_entropy <loss>=0.510625617729\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:13 INFO 139813082523456] #quality_metric: host=algo-1, epoch=112, train binary_f_1.000 <score>=0.780593618361\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 543.842077255249, \"sum\": 543.842077255249, \"min\": 543.842077255249}}, \"EndTime\": 1604871673.825167, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871673.280724}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:13 INFO 139813082523456] #progress_metric: host=algo-1, completed 56 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 10284, \"sum\": 10284.0, \"min\": 10284}, \"Total Records Seen\": {\"count\": 1, \"max\": 10235410, \"sum\": 10235410.0, \"min\": 10235410}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 114, \"sum\": 114.0, \"min\": 114}}, \"EndTime\": 1604871673.825358, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 112}, \"StartTime\": 1604871673.281301}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:13 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=166439.395211 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:13 INFO 139813082523456] #quality_metric: host=algo-1, epoch=113, batch=0 train binary_classification_accuracy <score>=0.721\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:13 INFO 139813082523456] #quality_metric: host=algo-1, epoch=113, batch=0 train binary_classification_cross_entropy <loss>=0.533914672852\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:13 INFO 139813082523456] #quality_metric: host=algo-1, epoch=113, batch=0 train binary_f_1.000 <score>=0.748874887489\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:14.387] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 228, \"duration\": 560, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:14 INFO 139813082523456] #quality_metric: host=algo-1, epoch=113, train binary_classification_accuracy <score>=0.75110989011\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:14 INFO 139813082523456] #quality_metric: host=algo-1, epoch=113, train binary_classification_cross_entropy <loss>=0.510316570575\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:14 INFO 139813082523456] #quality_metric: host=algo-1, epoch=113, train binary_f_1.000 <score>=0.78068383186\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 562.3831748962402, \"sum\": 562.3831748962402, \"min\": 562.3831748962402}}, \"EndTime\": 1604871674.388265, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871673.825225}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:14 INFO 139813082523456] #progress_metric: host=algo-1, completed 57 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 10375, \"sum\": 10375.0, \"min\": 10375}, \"Total Records Seen\": {\"count\": 1, \"max\": 10325980, \"sum\": 10325980.0, \"min\": 10325980}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 115, \"sum\": 115.0, \"min\": 115}}, \"EndTime\": 1604871674.388413, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 113}, \"StartTime\": 1604871673.825857}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:14 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=160973.282692 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:14 INFO 139813082523456] #quality_metric: host=algo-1, epoch=114, batch=0 train binary_classification_accuracy <score>=0.721\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:14 INFO 139813082523456] #quality_metric: host=algo-1, epoch=114, batch=0 train binary_classification_cross_entropy <loss>=0.533627319336\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:14 INFO 139813082523456] #quality_metric: host=algo-1, epoch=114, batch=0 train binary_f_1.000 <score>=0.748874887489\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:14.983] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 230, \"duration\": 593, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:14 INFO 139813082523456] #quality_metric: host=algo-1, epoch=114, train binary_classification_accuracy <score>=0.751296703297\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:14 INFO 139813082523456] #quality_metric: host=algo-1, epoch=114, train binary_classification_cross_entropy <loss>=0.510008244441\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:14 INFO 139813082523456] #quality_metric: host=algo-1, epoch=114, train binary_f_1.000 <score>=0.780833591571\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 594.9139595031738, \"sum\": 594.9139595031738, \"min\": 594.9139595031738}}, \"EndTime\": 1604871674.983829, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871674.38832}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:14 INFO 139813082523456] #progress_metric: host=algo-1, completed 57 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 10466, \"sum\": 10466.0, \"min\": 10466}, \"Total Records Seen\": {\"count\": 1, \"max\": 10416550, \"sum\": 10416550.0, \"min\": 10416550}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 116, \"sum\": 116.0, \"min\": 116}}, \"EndTime\": 1604871674.983985, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 114}, \"StartTime\": 1604871674.388889}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:14 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=152171.163077 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:14 INFO 139813082523456] #quality_metric: host=algo-1, epoch=115, batch=0 train binary_classification_accuracy <score>=0.72\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:14 INFO 139813082523456] #quality_metric: host=algo-1, epoch=115, batch=0 train binary_classification_cross_entropy <loss>=0.533339233398\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:14 INFO 139813082523456] #quality_metric: host=algo-1, epoch=115, batch=0 train binary_f_1.000 <score>=0.747747747748\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:15.556] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 232, \"duration\": 570, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:15 INFO 139813082523456] #quality_metric: host=algo-1, epoch=115, train binary_classification_accuracy <score>=0.751461538462\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:15 INFO 139813082523456] #quality_metric: host=algo-1, epoch=115, train binary_classification_cross_entropy <loss>=0.509700550792\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:15 INFO 139813082523456] #quality_metric: host=algo-1, epoch=115, train binary_f_1.000 <score>=0.781002178649\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 572.4868774414062, \"sum\": 572.4868774414062, \"min\": 572.4868774414062}}, \"EndTime\": 1604871675.557001, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871674.983886}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:15 INFO 139813082523456] #progress_metric: host=algo-1, completed 58 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 10557, \"sum\": 10557.0, \"min\": 10557}, \"Total Records Seen\": {\"count\": 1, \"max\": 10507120, \"sum\": 10507120.0, \"min\": 10507120}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 117, \"sum\": 117.0, \"min\": 117}}, \"EndTime\": 1604871675.557175, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 115}, \"StartTime\": 1604871674.984484}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:15 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=158119.683926 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:15 INFO 139813082523456] #quality_metric: host=algo-1, epoch=116, batch=0 train binary_classification_accuracy <score>=0.72\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:15 INFO 139813082523456] #quality_metric: host=algo-1, epoch=116, batch=0 train binary_classification_cross_entropy <loss>=0.533050415039\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:15 INFO 139813082523456] #quality_metric: host=algo-1, epoch=116, batch=0 train binary_f_1.000 <score>=0.747747747748\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:16.134] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 234, \"duration\": 576, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:16 INFO 139813082523456] #quality_metric: host=algo-1, epoch=116, train binary_classification_accuracy <score>=0.751538461538\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:16 INFO 139813082523456] #quality_metric: host=algo-1, epoch=116, train binary_classification_cross_entropy <loss>=0.509393407465\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:16 INFO 139813082523456] #quality_metric: host=algo-1, epoch=116, train binary_f_1.000 <score>=0.781072078702\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 577.725887298584, \"sum\": 577.725887298584, \"min\": 577.725887298584}}, \"EndTime\": 1604871676.135411, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871675.557056}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:16 INFO 139813082523456] #progress_metric: host=algo-1, completed 58 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 10648, \"sum\": 10648.0, \"min\": 10648}, \"Total Records Seen\": {\"count\": 1, \"max\": 10597690, \"sum\": 10597690.0, \"min\": 10597690}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 118, \"sum\": 118.0, \"min\": 118}}, \"EndTime\": 1604871676.135599, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 116}, \"StartTime\": 1604871675.557656}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:16 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=156683.009526 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:16 INFO 139813082523456] #quality_metric: host=algo-1, epoch=117, batch=0 train binary_classification_accuracy <score>=0.72\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:16 INFO 139813082523456] #quality_metric: host=algo-1, epoch=117, batch=0 train binary_classification_cross_entropy <loss>=0.532760864258\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:16 INFO 139813082523456] #quality_metric: host=algo-1, epoch=117, batch=0 train binary_f_1.000 <score>=0.747747747748\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:16.682] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 236, \"duration\": 545, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:16 INFO 139813082523456] #quality_metric: host=algo-1, epoch=117, train binary_classification_accuracy <score>=0.751703296703\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:16 INFO 139813082523456] #quality_metric: host=algo-1, epoch=117, train binary_classification_cross_entropy <loss>=0.509086735987\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:16 INFO 139813082523456] #quality_metric: host=algo-1, epoch=117, train binary_f_1.000 <score>=0.781236384761\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 546.9698905944824, \"sum\": 546.9698905944824, \"min\": 546.9698905944824}}, \"EndTime\": 1604871676.683088, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871676.135464}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:16 INFO 139813082523456] #progress_metric: host=algo-1, completed 59 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 10739, \"sum\": 10739.0, \"min\": 10739}, \"Total Records Seen\": {\"count\": 1, \"max\": 10688260, \"sum\": 10688260.0, \"min\": 10688260}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 119, \"sum\": 119.0, \"min\": 119}}, \"EndTime\": 1604871676.683238, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 117}, \"StartTime\": 1604871676.136093}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:16 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=165507.429215 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:16 INFO 139813082523456] #quality_metric: host=algo-1, epoch=118, batch=0 train binary_classification_accuracy <score>=0.721\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:16 INFO 139813082523456] #quality_metric: host=algo-1, epoch=118, batch=0 train binary_classification_cross_entropy <loss>=0.532470336914\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:16 INFO 139813082523456] #quality_metric: host=algo-1, epoch=118, batch=0 train binary_f_1.000 <score>=0.748422001803\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:17.247] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 238, \"duration\": 562, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:17 INFO 139813082523456] #quality_metric: host=algo-1, epoch=118, train binary_classification_accuracy <score>=0.751901098901\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:17 INFO 139813082523456] #quality_metric: host=algo-1, epoch=118, train binary_classification_cross_entropy <loss>=0.508780443129\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:17 INFO 139813082523456] #quality_metric: host=algo-1, epoch=118, train binary_f_1.000 <score>=0.781406427002\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 564.0451908111572, \"sum\": 564.0451908111572, \"min\": 564.0451908111572}}, \"EndTime\": 1604871677.247811, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871676.683145}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:17 INFO 139813082523456] #progress_metric: host=algo-1, completed 59 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 10830, \"sum\": 10830.0, \"min\": 10830}, \"Total Records Seen\": {\"count\": 1, \"max\": 10778830, \"sum\": 10778830.0, \"min\": 10778830}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 120, \"sum\": 120.0, \"min\": 120}}, \"EndTime\": 1604871677.247989, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 118}, \"StartTime\": 1604871676.683738}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:17 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=160486.088058 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:17 INFO 139813082523456] #quality_metric: host=algo-1, epoch=119, batch=0 train binary_classification_accuracy <score>=0.722\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:17 INFO 139813082523456] #quality_metric: host=algo-1, epoch=119, batch=0 train binary_classification_cross_entropy <loss>=0.532178833008\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:17 INFO 139813082523456] #quality_metric: host=algo-1, epoch=119, batch=0 train binary_f_1.000 <score>=0.74954954955\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:17.782] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 240, \"duration\": 532, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:17 INFO 139813082523456] #quality_metric: host=algo-1, epoch=119, train binary_classification_accuracy <score>=0.752021978022\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:17 INFO 139813082523456] #quality_metric: host=algo-1, epoch=119, train binary_classification_cross_entropy <loss>=0.508474466512\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:17 INFO 139813082523456] #quality_metric: host=algo-1, epoch=119, train binary_f_1.000 <score>=0.78151081505\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 534.5230102539062, \"sum\": 534.5230102539062, \"min\": 534.5230102539062}}, \"EndTime\": 1604871677.783024, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871677.247866}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:17 INFO 139813082523456] #progress_metric: host=algo-1, completed 60 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 10921, \"sum\": 10921.0, \"min\": 10921}, \"Total Records Seen\": {\"count\": 1, \"max\": 10869400, \"sum\": 10869400.0, \"min\": 10869400}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 121, \"sum\": 121.0, \"min\": 121}}, \"EndTime\": 1604871677.783172, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 119}, \"StartTime\": 1604871677.248475}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:17 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=169359.05264 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:17 INFO 139813082523456] #quality_metric: host=algo-1, epoch=120, batch=0 train binary_classification_accuracy <score>=0.722\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:17 INFO 139813082523456] #quality_metric: host=algo-1, epoch=120, batch=0 train binary_classification_cross_entropy <loss>=0.531886352539\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:17 INFO 139813082523456] #quality_metric: host=algo-1, epoch=120, batch=0 train binary_f_1.000 <score>=0.74954954955\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[34m[2020-11-08 21:41:18.345] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 242, \"duration\": 560, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:18 INFO 139813082523456] #quality_metric: host=algo-1, epoch=120, train binary_classification_accuracy <score>=0.752186813187\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:18 INFO 139813082523456] #quality_metric: host=algo-1, epoch=120, train binary_classification_cross_entropy <loss>=0.508168722635\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:18 INFO 139813082523456] #quality_metric: host=algo-1, epoch=120, train binary_f_1.000 <score>=0.781653934412\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 562.2141361236572, \"sum\": 562.2141361236572, \"min\": 562.2141361236572}}, \"EndTime\": 1604871678.345892, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871677.78308}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:18 INFO 139813082523456] #progress_metric: host=algo-1, completed 60 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 11012, \"sum\": 11012.0, \"min\": 11012}, \"Total Records Seen\": {\"count\": 1, \"max\": 10959970, \"sum\": 10959970.0, \"min\": 10959970}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 122, \"sum\": 122.0, \"min\": 122}}, \"EndTime\": 1604871678.346054, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 120}, \"StartTime\": 1604871677.783652}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:18 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=161017.08697 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:18 INFO 139813082523456] #quality_metric: host=algo-1, epoch=121, batch=0 train binary_classification_accuracy <score>=0.722\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:18 INFO 139813082523456] #quality_metric: host=algo-1, epoch=121, batch=0 train binary_classification_cross_entropy <loss>=0.531592773437\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:18 INFO 139813082523456] #quality_metric: host=algo-1, epoch=121, batch=0 train binary_f_1.000 <score>=0.74954954955\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:18.886] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 244, \"duration\": 538, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:18 INFO 139813082523456] #quality_metric: host=algo-1, epoch=121, train binary_classification_accuracy <score>=0.752285714286\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:18 INFO 139813082523456] #quality_metric: host=algo-1, epoch=121, train binary_classification_cross_entropy <loss>=0.50786314107\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:18 INFO 139813082523456] #quality_metric: host=algo-1, epoch=121, train binary_f_1.000 <score>=0.78173050854\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 540.1461124420166, \"sum\": 540.1461124420166, \"min\": 540.1461124420166}}, \"EndTime\": 1604871678.886727, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871678.345944}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:18 INFO 139813082523456] #progress_metric: host=algo-1, completed 61 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 11103, \"sum\": 11103.0, \"min\": 11103}, \"Total Records Seen\": {\"count\": 1, \"max\": 11050540, \"sum\": 11050540.0, \"min\": 11050540}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 123, \"sum\": 123.0, \"min\": 123}}, \"EndTime\": 1604871678.886878, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 121}, \"StartTime\": 1604871678.346555}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:18 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=167595.922442 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:18 INFO 139813082523456] #quality_metric: host=algo-1, epoch=122, batch=0 train binary_classification_accuracy <score>=0.722\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:18 INFO 139813082523456] #quality_metric: host=algo-1, epoch=122, batch=0 train binary_classification_cross_entropy <loss>=0.531298095703\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:18 INFO 139813082523456] #quality_metric: host=algo-1, epoch=122, batch=0 train binary_f_1.000 <score>=0.74954954955\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:19.431] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 246, \"duration\": 542, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:19 INFO 139813082523456] #quality_metric: host=algo-1, epoch=122, train binary_classification_accuracy <score>=0.75232967033\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:19 INFO 139813082523456] #quality_metric: host=algo-1, epoch=122, train binary_classification_cross_entropy <loss>=0.507557654077\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:19 INFO 139813082523456] #quality_metric: host=algo-1, epoch=122, train binary_f_1.000 <score>=0.781786143062\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 544.1470146179199, \"sum\": 544.1470146179199, \"min\": 544.1470146179199}}, \"EndTime\": 1604871679.431549, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871678.886782}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:19 INFO 139813082523456] #progress_metric: host=algo-1, completed 61 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 11194, \"sum\": 11194.0, \"min\": 11194}, \"Total Records Seen\": {\"count\": 1, \"max\": 11141110, \"sum\": 11141110.0, \"min\": 11141110}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 124, \"sum\": 124.0, \"min\": 124}}, \"EndTime\": 1604871679.4317, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 122}, \"StartTime\": 1604871678.887376}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:19 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=166363.589229 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:19 INFO 139813082523456] #quality_metric: host=algo-1, epoch=123, batch=0 train binary_classification_accuracy <score>=0.723\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:19 INFO 139813082523456] #quality_metric: host=algo-1, epoch=123, batch=0 train binary_classification_cross_entropy <loss>=0.531002197266\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:19 INFO 139813082523456] #quality_metric: host=algo-1, epoch=123, batch=0 train binary_f_1.000 <score>=0.750675067507\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:19.990] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 248, \"duration\": 557, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:19 INFO 139813082523456] #quality_metric: host=algo-1, epoch=123, train binary_classification_accuracy <score>=0.752516483516\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:19 INFO 139813082523456] #quality_metric: host=algo-1, epoch=123, train binary_classification_cross_entropy <loss>=0.507252190223\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:19 INFO 139813082523456] #quality_metric: host=algo-1, epoch=123, train binary_f_1.000 <score>=0.781969736575\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 558.7148666381836, \"sum\": 558.7148666381836, \"min\": 558.7148666381836}}, \"EndTime\": 1604871679.990951, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871679.431605}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:19 INFO 139813082523456] #progress_metric: host=algo-1, completed 62 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 11285, \"sum\": 11285.0, \"min\": 11285}, \"Total Records Seen\": {\"count\": 1, \"max\": 11231680, \"sum\": 11231680.0, \"min\": 11231680}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 125, \"sum\": 125.0, \"min\": 125}}, \"EndTime\": 1604871679.991168, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 123}, \"StartTime\": 1604871679.432207}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:19 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=162005.501118 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:20 INFO 139813082523456] #quality_metric: host=algo-1, epoch=124, batch=0 train binary_classification_accuracy <score>=0.723\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:20 INFO 139813082523456] #quality_metric: host=algo-1, epoch=124, batch=0 train binary_classification_cross_entropy <loss>=0.53070501709\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:20 INFO 139813082523456] #quality_metric: host=algo-1, epoch=124, batch=0 train binary_f_1.000 <score>=0.750675067507\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:20.568] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 250, \"duration\": 575, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:20 INFO 139813082523456] #quality_metric: host=algo-1, epoch=124, train binary_classification_accuracy <score>=0.752802197802\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:20 INFO 139813082523456] #quality_metric: host=algo-1, epoch=124, train binary_classification_cross_entropy <loss>=0.506946690486\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:20 INFO 139813082523456] #quality_metric: host=algo-1, epoch=124, train binary_f_1.000 <score>=0.782213013971\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 577.4831771850586, \"sum\": 577.4831771850586, \"min\": 577.4831771850586}}, \"EndTime\": 1604871680.569156, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871679.991014}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:20 INFO 139813082523456] #progress_metric: host=algo-1, completed 62 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 11376, \"sum\": 11376.0, \"min\": 11376}, \"Total Records Seen\": {\"count\": 1, \"max\": 11322250, \"sum\": 11322250.0, \"min\": 11322250}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 126, \"sum\": 126.0, \"min\": 126}}, \"EndTime\": 1604871680.56935, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 124}, \"StartTime\": 1604871679.991647}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:20 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=156747.855076 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:20 INFO 139813082523456] #quality_metric: host=algo-1, epoch=125, batch=0 train binary_classification_accuracy <score>=0.722\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:20 INFO 139813082523456] #quality_metric: host=algo-1, epoch=125, batch=0 train binary_classification_cross_entropy <loss>=0.530406555176\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:20 INFO 139813082523456] #quality_metric: host=algo-1, epoch=125, batch=0 train binary_f_1.000 <score>=0.75\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:21.126] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 252, \"duration\": 555, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:21 INFO 139813082523456] #quality_metric: host=algo-1, epoch=125, train binary_classification_accuracy <score>=0.752923076923\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:21 INFO 139813082523456] #quality_metric: host=algo-1, epoch=125, train binary_classification_cross_entropy <loss>=0.506641085782\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:21 INFO 139813082523456] #quality_metric: host=algo-1, epoch=125, train binary_f_1.000 <score>=0.782321618743\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 557.3430061340332, \"sum\": 557.3430061340332, \"min\": 557.3430061340332}}, \"EndTime\": 1604871681.127205, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871680.569219}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:21 INFO 139813082523456] #progress_metric: host=algo-1, completed 63 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 11467, \"sum\": 11467.0, \"min\": 11467}, \"Total Records Seen\": {\"count\": 1, \"max\": 11412820, \"sum\": 11412820.0, \"min\": 11412820}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 127, \"sum\": 127.0, \"min\": 127}}, \"EndTime\": 1604871681.127356, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 125}, \"StartTime\": 1604871680.569837}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:21 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=162424.785287 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:21 INFO 139813082523456] #quality_metric: host=algo-1, epoch=126, batch=0 train binary_classification_accuracy <score>=0.722\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:21 INFO 139813082523456] #quality_metric: host=algo-1, epoch=126, batch=0 train binary_classification_cross_entropy <loss>=0.530106689453\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:21 INFO 139813082523456] #quality_metric: host=algo-1, epoch=126, batch=0 train binary_f_1.000 <score>=0.750448833034\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:21.678] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 254, \"duration\": 549, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:21 INFO 139813082523456] #quality_metric: host=algo-1, epoch=126, train binary_classification_accuracy <score>=0.753043956044\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:21 INFO 139813082523456] #quality_metric: host=algo-1, epoch=126, train binary_classification_cross_entropy <loss>=0.506335320106\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:21 INFO 139813082523456] #quality_metric: host=algo-1, epoch=126, train binary_f_1.000 <score>=0.782459706694\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 551.3219833374023, \"sum\": 551.3219833374023, \"min\": 551.3219833374023}}, \"EndTime\": 1604871681.679201, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871681.127261}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:21 INFO 139813082523456] #progress_metric: host=algo-1, completed 63 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 11558, \"sum\": 11558.0, \"min\": 11558}, \"Total Records Seen\": {\"count\": 1, \"max\": 11503390, \"sum\": 11503390.0, \"min\": 11503390}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 128, \"sum\": 128.0, \"min\": 128}}, \"EndTime\": 1604871681.679402, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 126}, \"StartTime\": 1604871681.127851}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:21 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=164178.682545 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:21 INFO 139813082523456] #quality_metric: host=algo-1, epoch=127, batch=0 train binary_classification_accuracy <score>=0.724\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:21 INFO 139813082523456] #quality_metric: host=algo-1, epoch=127, batch=0 train binary_classification_cross_entropy <loss>=0.529805419922\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:21 INFO 139813082523456] #quality_metric: host=algo-1, epoch=127, batch=0 train binary_f_1.000 <score>=0.752244165171\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:22.225] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 256, \"duration\": 544, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:22 INFO 139813082523456] #quality_metric: host=algo-1, epoch=127, train binary_classification_accuracy <score>=0.753120879121\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:22 INFO 139813082523456] #quality_metric: host=algo-1, epoch=127, train binary_classification_cross_entropy <loss>=0.506029327393\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:22 INFO 139813082523456] #quality_metric: host=algo-1, epoch=127, train binary_f_1.000 <score>=0.782521151575\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 546.299934387207, \"sum\": 546.299934387207, \"min\": 546.299934387207}}, \"EndTime\": 1604871682.226233, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871681.679266}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:22 INFO 139813082523456] #progress_metric: host=algo-1, completed 64 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 11649, \"sum\": 11649.0, \"min\": 11649}, \"Total Records Seen\": {\"count\": 1, \"max\": 11593960, \"sum\": 11593960.0, \"min\": 11593960}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 129, \"sum\": 129.0, \"min\": 129}}, \"EndTime\": 1604871682.22638, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 127}, \"StartTime\": 1604871681.679906}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:22 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=165708.85875 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:22 INFO 139813082523456] #quality_metric: host=algo-1, epoch=128, batch=0 train binary_classification_accuracy <score>=0.724\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:22 INFO 139813082523456] #quality_metric: host=algo-1, epoch=128, batch=0 train binary_classification_cross_entropy <loss>=0.529502746582\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:22 INFO 139813082523456] #quality_metric: host=algo-1, epoch=128, batch=0 train binary_f_1.000 <score>=0.752244165171\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:22.767] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 258, \"duration\": 539, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:22 INFO 139813082523456] #quality_metric: host=algo-1, epoch=128, train binary_classification_accuracy <score>=0.753274725275\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:22 INFO 139813082523456] #quality_metric: host=algo-1, epoch=128, train binary_classification_cross_entropy <loss>=0.505723054991\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:22 INFO 139813082523456] #quality_metric: host=algo-1, epoch=128, train binary_f_1.000 <score>=0.782665092057\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 541.1109924316406, \"sum\": 541.1109924316406, \"min\": 541.1109924316406}}, \"EndTime\": 1604871682.768017, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871682.226289}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:22 INFO 139813082523456] #progress_metric: host=algo-1, completed 64 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 11740, \"sum\": 11740.0, \"min\": 11740}, \"Total Records Seen\": {\"count\": 1, \"max\": 11684530, \"sum\": 11684530.0, \"min\": 11684530}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 130, \"sum\": 130.0, \"min\": 130}}, \"EndTime\": 1604871682.768164, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 128}, \"StartTime\": 1604871682.226881}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:22 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=167296.850014 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:22 INFO 139813082523456] #quality_metric: host=algo-1, epoch=129, batch=0 train binary_classification_accuracy <score>=0.723\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:22 INFO 139813082523456] #quality_metric: host=algo-1, epoch=129, batch=0 train binary_classification_cross_entropy <loss>=0.529198486328\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:22 INFO 139813082523456] #quality_metric: host=algo-1, epoch=129, batch=0 train binary_f_1.000 <score>=0.751123090746\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:23.324] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 260, \"duration\": 554, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:23 INFO 139813082523456] #quality_metric: host=algo-1, epoch=129, train binary_classification_accuracy <score>=0.753450549451\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:23 INFO 139813082523456] #quality_metric: host=algo-1, epoch=129, train binary_classification_cross_entropy <loss>=0.505416444548\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:23 INFO 139813082523456] #quality_metric: host=algo-1, epoch=129, train binary_f_1.000 <score>=0.782798946716\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 556.1239719390869, \"sum\": 556.1239719390869, \"min\": 556.1239719390869}}, \"EndTime\": 1604871683.32481, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871682.768071}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:23 INFO 139813082523456] #progress_metric: host=algo-1, completed 65 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 11831, \"sum\": 11831.0, \"min\": 11831}, \"Total Records Seen\": {\"count\": 1, \"max\": 11775100, \"sum\": 11775100.0, \"min\": 11775100}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 131, \"sum\": 131.0, \"min\": 131}}, \"EndTime\": 1604871683.32501, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 129}, \"StartTime\": 1604871682.768661}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:23 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=162764.191392 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:23 INFO 139813082523456] #quality_metric: host=algo-1, epoch=130, batch=0 train binary_classification_accuracy <score>=0.723\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:23 INFO 139813082523456] #quality_metric: host=algo-1, epoch=130, batch=0 train binary_classification_cross_entropy <loss>=0.52889276123\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:23 INFO 139813082523456] #quality_metric: host=algo-1, epoch=130, batch=0 train binary_f_1.000 <score>=0.751123090746\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:23.879] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 262, \"duration\": 552, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:23 INFO 139813082523456] #quality_metric: host=algo-1, epoch=130, train binary_classification_accuracy <score>=0.753615384615\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:23 INFO 139813082523456] #quality_metric: host=algo-1, epoch=130, train binary_classification_cross_entropy <loss>=0.50510944509\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:23 INFO 139813082523456] #quality_metric: host=algo-1, epoch=130, train binary_f_1.000 <score>=0.78295466646\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 554.1260242462158, \"sum\": 554.1260242462158, \"min\": 554.1260242462158}}, \"EndTime\": 1604871683.879633, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871683.324865}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:23 INFO 139813082523456] #progress_metric: host=algo-1, completed 65 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 11922, \"sum\": 11922.0, \"min\": 11922}, \"Total Records Seen\": {\"count\": 1, \"max\": 11865670, \"sum\": 11865670.0, \"min\": 11865670}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 132, \"sum\": 132.0, \"min\": 132}}, \"EndTime\": 1604871683.879784, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 130}, \"StartTime\": 1604871683.325481}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:23 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=163370.517763 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:23 INFO 139813082523456] #quality_metric: host=algo-1, epoch=131, batch=0 train binary_classification_accuracy <score>=0.724\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:23 INFO 139813082523456] #quality_metric: host=algo-1, epoch=131, batch=0 train binary_classification_cross_entropy <loss>=0.528585449219\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:23 INFO 139813082523456] #quality_metric: host=algo-1, epoch=131, batch=0 train binary_f_1.000 <score>=0.751798561151\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:24.445] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 264, \"duration\": 563, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:24 INFO 139813082523456] #quality_metric: host=algo-1, epoch=131, train binary_classification_accuracy <score>=0.753802197802\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:24 INFO 139813082523456] #quality_metric: host=algo-1, epoch=131, train binary_classification_cross_entropy <loss>=0.504802002288\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:24 INFO 139813082523456] #quality_metric: host=algo-1, epoch=131, train binary_f_1.000 <score>=0.783142326158\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 565.3519630432129, \"sum\": 565.3519630432129, \"min\": 565.3519630432129}}, \"EndTime\": 1604871684.445659, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871683.879689}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:24 INFO 139813082523456] #progress_metric: host=algo-1, completed 66 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 12013, \"sum\": 12013.0, \"min\": 12013}, \"Total Records Seen\": {\"count\": 1, \"max\": 11956240, \"sum\": 11956240.0, \"min\": 11956240}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 133, \"sum\": 133.0, \"min\": 133}}, \"EndTime\": 1604871684.445872, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 131}, \"StartTime\": 1604871683.880277}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:24 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=160101.702798 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:24 INFO 139813082523456] #quality_metric: host=algo-1, epoch=132, batch=0 train binary_classification_accuracy <score>=0.725\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:24 INFO 139813082523456] #quality_metric: host=algo-1, epoch=132, batch=0 train binary_classification_cross_entropy <loss>=0.528276428223\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:24 INFO 139813082523456] #quality_metric: host=algo-1, epoch=132, batch=0 train binary_f_1.000 <score>=0.752920035939\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:25.016] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 266, \"duration\": 568, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:25 INFO 139813082523456] #quality_metric: host=algo-1, epoch=132, train binary_classification_accuracy <score>=0.753857142857\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:25 INFO 139813082523456] #quality_metric: host=algo-1, epoch=132, train binary_classification_cross_entropy <loss>=0.504494068523\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:25 INFO 139813082523456] #quality_metric: host=algo-1, epoch=132, train binary_f_1.000 <score>=0.783184427301\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 570.5959796905518, \"sum\": 570.5959796905518, \"min\": 570.5959796905518}}, \"EndTime\": 1604871685.017014, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871684.445721}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:25 INFO 139813082523456] #progress_metric: host=algo-1, completed 66 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 12104, \"sum\": 12104.0, \"min\": 12104}, \"Total Records Seen\": {\"count\": 1, \"max\": 12046810, \"sum\": 12046810.0, \"min\": 12046810}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 134, \"sum\": 134.0, \"min\": 134}}, \"EndTime\": 1604871685.017205, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 132}, \"StartTime\": 1604871684.44639}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:25 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=158638.09483 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:25 INFO 139813082523456] #quality_metric: host=algo-1, epoch=133, batch=0 train binary_classification_accuracy <score>=0.725\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:25 INFO 139813082523456] #quality_metric: host=algo-1, epoch=133, batch=0 train binary_classification_cross_entropy <loss>=0.527965759277\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:25 INFO 139813082523456] #quality_metric: host=algo-1, epoch=133, batch=0 train binary_f_1.000 <score>=0.752920035939\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:25.578] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 268, \"duration\": 559, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:25 INFO 139813082523456] #quality_metric: host=algo-1, epoch=133, train binary_classification_accuracy <score>=0.754032967033\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:25 INFO 139813082523456] #quality_metric: host=algo-1, epoch=133, train binary_classification_cross_entropy <loss>=0.50418559416\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:25 INFO 139813082523456] #quality_metric: host=algo-1, epoch=133, train binary_f_1.000 <score>=0.783314133034\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 561.2678527832031, \"sum\": 561.2678527832031, \"min\": 561.2678527832031}}, \"EndTime\": 1604871685.578999, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871685.017068}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:25 INFO 139813082523456] #progress_metric: host=algo-1, completed 67 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 12195, \"sum\": 12195.0, \"min\": 12195}, \"Total Records Seen\": {\"count\": 1, \"max\": 12137380, \"sum\": 12137380.0, \"min\": 12137380}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 135, \"sum\": 135.0, \"min\": 135}}, \"EndTime\": 1604871685.579217, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 133}, \"StartTime\": 1604871685.017701}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:25 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=161264.187643 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:25 INFO 139813082523456] #quality_metric: host=algo-1, epoch=134, batch=0 train binary_classification_accuracy <score>=0.724\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:25 INFO 139813082523456] #quality_metric: host=algo-1, epoch=134, batch=0 train binary_classification_cross_entropy <loss>=0.527653381348\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:25 INFO 139813082523456] #quality_metric: host=algo-1, epoch=134, batch=0 train binary_f_1.000 <score>=0.751798561151\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:26.147] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 270, \"duration\": 566, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:26 INFO 139813082523456] #quality_metric: host=algo-1, epoch=134, train binary_classification_accuracy <score>=0.754230769231\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:26 INFO 139813082523456] #quality_metric: host=algo-1, epoch=134, train binary_classification_cross_entropy <loss>=0.503876530909\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:26 INFO 139813082523456] #quality_metric: host=algo-1, epoch=134, train binary_f_1.000 <score>=0.783475811058\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 567.8281784057617, \"sum\": 567.8281784057617, \"min\": 567.8281784057617}}, \"EndTime\": 1604871686.147612, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871685.579066}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:26 INFO 139813082523456] #progress_metric: host=algo-1, completed 67 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 12286, \"sum\": 12286.0, \"min\": 12286}, \"Total Records Seen\": {\"count\": 1, \"max\": 12227950, \"sum\": 12227950.0, \"min\": 12227950}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 136, \"sum\": 136.0, \"min\": 136}}, \"EndTime\": 1604871686.147761, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 134}, \"StartTime\": 1604871685.579756}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:26 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=159428.907473 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:26 INFO 139813082523456] #quality_metric: host=algo-1, epoch=135, batch=0 train binary_classification_accuracy <score>=0.724\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:26 INFO 139813082523456] #quality_metric: host=algo-1, epoch=135, batch=0 train binary_classification_cross_entropy <loss>=0.527339233398\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:26 INFO 139813082523456] #quality_metric: host=algo-1, epoch=135, batch=0 train binary_f_1.000 <score>=0.751798561151\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:26.699] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 272, \"duration\": 549, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:26 INFO 139813082523456] #quality_metric: host=algo-1, epoch=135, train binary_classification_accuracy <score>=0.754406593407\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:26 INFO 139813082523456] #quality_metric: host=algo-1, epoch=135, train binary_classification_cross_entropy <loss>=0.503566836514\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:26 INFO 139813082523456] #quality_metric: host=algo-1, epoch=135, train binary_f_1.000 <score>=0.783634902656\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 551.5859127044678, \"sum\": 551.5859127044678, \"min\": 551.5859127044678}}, \"EndTime\": 1604871686.699871, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871686.147668}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:26 INFO 139813082523456] #progress_metric: host=algo-1, completed 68 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 12377, \"sum\": 12377.0, \"min\": 12377}, \"Total Records Seen\": {\"count\": 1, \"max\": 12318520, \"sum\": 12318520.0, \"min\": 12318520}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 137, \"sum\": 137.0, \"min\": 137}}, \"EndTime\": 1604871686.700017, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 135}, \"StartTime\": 1604871686.14826}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:26 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=164121.86644 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:26 INFO 139813082523456] #quality_metric: host=algo-1, epoch=136, batch=0 train binary_classification_accuracy <score>=0.724\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:26 INFO 139813082523456] #quality_metric: host=algo-1, epoch=136, batch=0 train binary_classification_cross_entropy <loss>=0.52702331543\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:26 INFO 139813082523456] #quality_metric: host=algo-1, epoch=136, batch=0 train binary_f_1.000 <score>=0.751798561151\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:27.246] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 274, \"duration\": 545, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:27 INFO 139813082523456] #quality_metric: host=algo-1, epoch=136, train binary_classification_accuracy <score>=0.754527472527\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:27 INFO 139813082523456] #quality_metric: host=algo-1, epoch=136, train binary_classification_cross_entropy <loss>=0.503256458324\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:27 INFO 139813082523456] #quality_metric: host=algo-1, epoch=136, train binary_f_1.000 <score>=0.783714175058\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 546.6709136962891, \"sum\": 546.6709136962891, \"min\": 546.6709136962891}}, \"EndTime\": 1604871687.247202, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871686.699926}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:27 INFO 139813082523456] #progress_metric: host=algo-1, completed 68 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 12468, \"sum\": 12468.0, \"min\": 12468}, \"Total Records Seen\": {\"count\": 1, \"max\": 12409090, \"sum\": 12409090.0, \"min\": 12409090}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 138, \"sum\": 138.0, \"min\": 138}}, \"EndTime\": 1604871687.247419, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 136}, \"StartTime\": 1604871686.700502}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:27 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=165568.023124 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:27 INFO 139813082523456] #quality_metric: host=algo-1, epoch=137, batch=0 train binary_classification_accuracy <score>=0.725\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:27 INFO 139813082523456] #quality_metric: host=algo-1, epoch=137, batch=0 train binary_classification_cross_entropy <loss>=0.526705566406\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:27 INFO 139813082523456] #quality_metric: host=algo-1, epoch=137, batch=0 train binary_f_1.000 <score>=0.752920035939\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:27.790] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 276, \"duration\": 541, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:27 INFO 139813082523456] #quality_metric: host=algo-1, epoch=137, train binary_classification_accuracy <score>=0.754714285714\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:27 INFO 139813082523456] #quality_metric: host=algo-1, epoch=137, train binary_classification_cross_entropy <loss>=0.502945361127\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:27 INFO 139813082523456] #quality_metric: host=algo-1, epoch=137, train binary_f_1.000 <score>=0.783897607683\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 543.065071105957, \"sum\": 543.065071105957, \"min\": 543.065071105957}}, \"EndTime\": 1604871687.791067, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871687.247266}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:27 INFO 139813082523456] #progress_metric: host=algo-1, completed 69 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 12559, \"sum\": 12559.0, \"min\": 12559}, \"Total Records Seen\": {\"count\": 1, \"max\": 12499660, \"sum\": 12499660.0, \"min\": 12499660}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 139, \"sum\": 139.0, \"min\": 139}}, \"EndTime\": 1604871687.791224, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 137}, \"StartTime\": 1604871687.247977}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:27 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=166693.703091 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:27 INFO 139813082523456] #quality_metric: host=algo-1, epoch=138, batch=0 train binary_classification_accuracy <score>=0.725\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:27 INFO 139813082523456] #quality_metric: host=algo-1, epoch=138, batch=0 train binary_classification_cross_entropy <loss>=0.526385986328\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:27 INFO 139813082523456] #quality_metric: host=algo-1, epoch=138, batch=0 train binary_f_1.000 <score>=0.752920035939\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[34m[2020-11-08 21:41:28.352] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 278, \"duration\": 559, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:28 INFO 139813082523456] #quality_metric: host=algo-1, epoch=138, train binary_classification_accuracy <score>=0.75489010989\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:28 INFO 139813082523456] #quality_metric: host=algo-1, epoch=138, train binary_classification_cross_entropy <loss>=0.502633497301\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:28 INFO 139813082523456] #quality_metric: host=algo-1, epoch=138, train binary_f_1.000 <score>=0.784027421401\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 561.1100196838379, \"sum\": 561.1100196838379, \"min\": 561.1100196838379}}, \"EndTime\": 1604871688.352844, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871687.791124}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:28 INFO 139813082523456] #progress_metric: host=algo-1, completed 69 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 12650, \"sum\": 12650.0, \"min\": 12650}, \"Total Records Seen\": {\"count\": 1, \"max\": 12590230, \"sum\": 12590230.0, \"min\": 12590230}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 140, \"sum\": 140.0, \"min\": 140}}, \"EndTime\": 1604871688.353031, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 138}, \"StartTime\": 1604871687.791705}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:28 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=161320.206692 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:28 INFO 139813082523456] #quality_metric: host=algo-1, epoch=139, batch=0 train binary_classification_accuracy <score>=0.725\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:28 INFO 139813082523456] #quality_metric: host=algo-1, epoch=139, batch=0 train binary_classification_cross_entropy <loss>=0.526064453125\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:28 INFO 139813082523456] #quality_metric: host=algo-1, epoch=139, batch=0 train binary_f_1.000 <score>=0.752920035939\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:28.889] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 280, \"duration\": 534, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:28 INFO 139813082523456] #quality_metric: host=algo-1, epoch=139, train binary_classification_accuracy <score>=0.755208791209\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:28 INFO 139813082523456] #quality_metric: host=algo-1, epoch=139, train binary_classification_cross_entropy <loss>=0.502320833982\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:28 INFO 139813082523456] #quality_metric: host=algo-1, epoch=139, train binary_f_1.000 <score>=0.784301954025\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 536.0150337219238, \"sum\": 536.0150337219238, \"min\": 536.0150337219238}}, \"EndTime\": 1604871688.889569, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871688.352901}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:28 INFO 139813082523456] #progress_metric: host=algo-1, completed 70 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 12741, \"sum\": 12741.0, \"min\": 12741}, \"Total Records Seen\": {\"count\": 1, \"max\": 12680800, \"sum\": 12680800.0, \"min\": 12680800}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 141, \"sum\": 141.0, \"min\": 141}}, \"EndTime\": 1604871688.889765, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 139}, \"StartTime\": 1604871688.353526}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:28 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=168867.664971 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:28 INFO 139813082523456] #quality_metric: host=algo-1, epoch=140, batch=0 train binary_classification_accuracy <score>=0.726\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:28 INFO 139813082523456] #quality_metric: host=algo-1, epoch=140, batch=0 train binary_classification_cross_entropy <loss>=0.525741088867\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:28 INFO 139813082523456] #quality_metric: host=algo-1, epoch=140, batch=0 train binary_f_1.000 <score>=0.754039497307\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:29.443] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 282, \"duration\": 552, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:29 INFO 139813082523456] #quality_metric: host=algo-1, epoch=140, train binary_classification_accuracy <score>=0.75532967033\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:29 INFO 139813082523456] #quality_metric: host=algo-1, epoch=140, train binary_classification_cross_entropy <loss>=0.502007327572\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:29 INFO 139813082523456] #quality_metric: host=algo-1, epoch=140, train binary_f_1.000 <score>=0.784418904133\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 553.9240837097168, \"sum\": 553.9240837097168, \"min\": 553.9240837097168}}, \"EndTime\": 1604871689.444206, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871688.889633}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:29 INFO 139813082523456] #progress_metric: host=algo-1, completed 70 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 12832, \"sum\": 12832.0, \"min\": 12832}, \"Total Records Seen\": {\"count\": 1, \"max\": 12771370, \"sum\": 12771370.0, \"min\": 12771370}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 142, \"sum\": 142.0, \"min\": 142}}, \"EndTime\": 1604871689.444353, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 140}, \"StartTime\": 1604871688.890258}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:29 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=163430.892705 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:29 INFO 139813082523456] #quality_metric: host=algo-1, epoch=141, batch=0 train binary_classification_accuracy <score>=0.726\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:29 INFO 139813082523456] #quality_metric: host=algo-1, epoch=141, batch=0 train binary_classification_cross_entropy <loss>=0.525415710449\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:29 INFO 139813082523456] #quality_metric: host=algo-1, epoch=141, batch=0 train binary_f_1.000 <score>=0.754039497307\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:30.004] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 284, \"duration\": 558, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:30 INFO 139813082523456] #quality_metric: host=algo-1, epoch=141, train binary_classification_accuracy <score>=0.755538461538\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:30 INFO 139813082523456] #quality_metric: host=algo-1, epoch=141, train binary_classification_cross_entropy <loss>=0.501692944202\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:30 INFO 139813082523456] #quality_metric: host=algo-1, epoch=141, train binary_f_1.000 <score>=0.784613299253\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 560.014009475708, \"sum\": 560.014009475708, \"min\": 560.014009475708}}, \"EndTime\": 1604871690.004868, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871689.444262}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:30 INFO 139813082523456] #progress_metric: host=algo-1, completed 71 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 12923, \"sum\": 12923.0, \"min\": 12923}, \"Total Records Seen\": {\"count\": 1, \"max\": 12861940, \"sum\": 12861940.0, \"min\": 12861940}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 143, \"sum\": 143.0, \"min\": 143}}, \"EndTime\": 1604871690.005019, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 141}, \"StartTime\": 1604871689.444828}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:30 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=161652.462195 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:30 INFO 139813082523456] #quality_metric: host=algo-1, epoch=142, batch=0 train binary_classification_accuracy <score>=0.727\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:30 INFO 139813082523456] #quality_metric: host=algo-1, epoch=142, batch=0 train binary_classification_cross_entropy <loss>=0.525088378906\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:30 INFO 139813082523456] #quality_metric: host=algo-1, epoch=142, batch=0 train binary_f_1.000 <score>=0.755156950673\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:30.554] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 286, \"duration\": 548, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:30 INFO 139813082523456] #quality_metric: host=algo-1, epoch=142, train binary_classification_accuracy <score>=0.755736263736\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:30 INFO 139813082523456] #quality_metric: host=algo-1, epoch=142, train binary_classification_cross_entropy <loss>=0.501377638597\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:30 INFO 139813082523456] #quality_metric: host=algo-1, epoch=142, train binary_f_1.000 <score>=0.784787576004\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 549.7210025787354, \"sum\": 549.7210025787354, \"min\": 549.7210025787354}}, \"EndTime\": 1604871690.555265, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871690.004926}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:30 INFO 139813082523456] #progress_metric: host=algo-1, completed 71 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 13014, \"sum\": 13014.0, \"min\": 13014}, \"Total Records Seen\": {\"count\": 1, \"max\": 12952510, \"sum\": 12952510.0, \"min\": 12952510}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 144, \"sum\": 144.0, \"min\": 144}}, \"EndTime\": 1604871690.555452, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 142}, \"StartTime\": 1604871690.005517}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:30 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=164664.033197 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:30 INFO 139813082523456] #quality_metric: host=algo-1, epoch=143, batch=0 train binary_classification_accuracy <score>=0.728\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:30 INFO 139813082523456] #quality_metric: host=algo-1, epoch=143, batch=0 train binary_classification_cross_entropy <loss>=0.524759033203\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:30 INFO 139813082523456] #quality_metric: host=algo-1, epoch=143, batch=0 train binary_f_1.000 <score>=0.756272401434\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:31.114] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 288, \"duration\": 557, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:31 INFO 139813082523456] #quality_metric: host=algo-1, epoch=143, train binary_classification_accuracy <score>=0.755967032967\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:31 INFO 139813082523456] #quality_metric: host=algo-1, epoch=143, train binary_classification_cross_entropy <loss>=0.501061382252\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:31 INFO 139813082523456] #quality_metric: host=algo-1, epoch=143, train binary_f_1.000 <score>=0.784984653518\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 559.2479705810547, \"sum\": 559.2479705810547, \"min\": 559.2479705810547}}, \"EndTime\": 1604871691.115176, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871690.555326}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:31 INFO 139813082523456] #progress_metric: host=algo-1, completed 72 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 13105, \"sum\": 13105.0, \"min\": 13105}, \"Total Records Seen\": {\"count\": 1, \"max\": 13043080, \"sum\": 13043080.0, \"min\": 13043080}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 145, \"sum\": 145.0, \"min\": 145}}, \"EndTime\": 1604871691.115382, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 143}, \"StartTime\": 1604871690.555901}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:31 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=161854.128298 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:31 INFO 139813082523456] #quality_metric: host=algo-1, epoch=144, batch=0 train binary_classification_accuracy <score>=0.729\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:31 INFO 139813082523456] #quality_metric: host=algo-1, epoch=144, batch=0 train binary_classification_cross_entropy <loss>=0.524427612305\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:31 INFO 139813082523456] #quality_metric: host=algo-1, epoch=144, batch=0 train binary_f_1.000 <score>=0.756950672646\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:31.672] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 290, \"duration\": 555, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:31 INFO 139813082523456] #quality_metric: host=algo-1, epoch=144, train binary_classification_accuracy <score>=0.756230769231\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:31 INFO 139813082523456] #quality_metric: host=algo-1, epoch=144, train binary_classification_cross_entropy <loss>=0.500744144649\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:31 INFO 139813082523456] #quality_metric: host=algo-1, epoch=144, train binary_f_1.000 <score>=0.78522534734\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 557.5270652770996, \"sum\": 557.5270652770996, \"min\": 557.5270652770996}}, \"EndTime\": 1604871691.67342, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871691.115232}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:31 INFO 139813082523456] #progress_metric: host=algo-1, completed 72 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 13196, \"sum\": 13196.0, \"min\": 13196}, \"Total Records Seen\": {\"count\": 1, \"max\": 13133650, \"sum\": 13133650.0, \"min\": 13133650}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 146, \"sum\": 146.0, \"min\": 146}}, \"EndTime\": 1604871691.673625, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 144}, \"StartTime\": 1604871691.115864}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:31 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=162352.868918 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:31 INFO 139813082523456] #quality_metric: host=algo-1, epoch=145, batch=0 train binary_classification_accuracy <score>=0.729\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:31 INFO 139813082523456] #quality_metric: host=algo-1, epoch=145, batch=0 train binary_classification_cross_entropy <loss>=0.524094177246\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:31 INFO 139813082523456] #quality_metric: host=algo-1, epoch=145, batch=0 train binary_f_1.000 <score>=0.756950672646\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:32.234] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 292, \"duration\": 559, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:32 INFO 139813082523456] #quality_metric: host=algo-1, epoch=145, train binary_classification_accuracy <score>=0.756384615385\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:32 INFO 139813082523456] #quality_metric: host=algo-1, epoch=145, train binary_classification_cross_entropy <loss>=0.50042588454\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:32 INFO 139813082523456] #quality_metric: host=algo-1, epoch=145, train binary_f_1.000 <score>=0.785365050781\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 561.0461235046387, \"sum\": 561.0461235046387, \"min\": 561.0461235046387}}, \"EndTime\": 1604871692.235203, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871691.673479}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:32 INFO 139813082523456] #progress_metric: host=algo-1, completed 73 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 13287, \"sum\": 13287.0, \"min\": 13287}, \"Total Records Seen\": {\"count\": 1, \"max\": 13224220, \"sum\": 13224220.0, \"min\": 13224220}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 147, \"sum\": 147.0, \"min\": 147}}, \"EndTime\": 1604871692.235399, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 145}, \"StartTime\": 1604871691.67413}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:32 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=161341.515101 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:32 INFO 139813082523456] #quality_metric: host=algo-1, epoch=146, batch=0 train binary_classification_accuracy <score>=0.729\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:32 INFO 139813082523456] #quality_metric: host=algo-1, epoch=146, batch=0 train binary_classification_cross_entropy <loss>=0.523758666992\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:32 INFO 139813082523456] #quality_metric: host=algo-1, epoch=146, batch=0 train binary_f_1.000 <score>=0.756950672646\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:32.793] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 294, \"duration\": 556, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:32 INFO 139813082523456] #quality_metric: host=algo-1, epoch=146, train binary_classification_accuracy <score>=0.75667032967\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:32 INFO 139813082523456] #quality_metric: host=algo-1, epoch=146, train binary_classification_cross_entropy <loss>=0.500106566041\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:32 INFO 139813082523456] #quality_metric: host=algo-1, epoch=146, train binary_f_1.000 <score>=0.785625078661\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 557.8570365905762, \"sum\": 557.8570365905762, \"min\": 557.8570365905762}}, \"EndTime\": 1604871692.793764, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871692.235262}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:32 INFO 139813082523456] #progress_metric: host=algo-1, completed 73 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 13378, \"sum\": 13378.0, \"min\": 13378}, \"Total Records Seen\": {\"count\": 1, \"max\": 13314790, \"sum\": 13314790.0, \"min\": 13314790}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 148, \"sum\": 148.0, \"min\": 148}}, \"EndTime\": 1604871692.793962, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 146}, \"StartTime\": 1604871692.235882}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:32 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=162254.538298 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:32 INFO 139813082523456] #quality_metric: host=algo-1, epoch=147, batch=0 train binary_classification_accuracy <score>=0.729\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:32 INFO 139813082523456] #quality_metric: host=algo-1, epoch=147, batch=0 train binary_classification_cross_entropy <loss>=0.523420959473\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:32 INFO 139813082523456] #quality_metric: host=algo-1, epoch=147, batch=0 train binary_f_1.000 <score>=0.756950672646\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:33.359] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 296, \"duration\": 564, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:33 INFO 139813082523456] #quality_metric: host=algo-1, epoch=147, train binary_classification_accuracy <score>=0.756604395604\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:33 INFO 139813082523456] #quality_metric: host=algo-1, epoch=147, train binary_classification_cross_entropy <loss>=0.499786166348\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:33 INFO 139813082523456] #quality_metric: host=algo-1, epoch=147, train binary_f_1.000 <score>=0.785579445676\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 565.8481121063232, \"sum\": 565.8481121063232, \"min\": 565.8481121063232}}, \"EndTime\": 1604871693.360376, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871692.793824}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:33 INFO 139813082523456] #progress_metric: host=algo-1, completed 74 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 13469, \"sum\": 13469.0, \"min\": 13469}, \"Total Records Seen\": {\"count\": 1, \"max\": 13405360, \"sum\": 13405360.0, \"min\": 13405360}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 149, \"sum\": 149.0, \"min\": 149}}, \"EndTime\": 1604871693.360589, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 147}, \"StartTime\": 1604871692.794498}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:33 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=159959.792272 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:33 INFO 139813082523456] #quality_metric: host=algo-1, epoch=148, batch=0 train binary_classification_accuracy <score>=0.728\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:33 INFO 139813082523456] #quality_metric: host=algo-1, epoch=148, batch=0 train binary_classification_cross_entropy <loss>=0.523081176758\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:33 INFO 139813082523456] #quality_metric: host=algo-1, epoch=148, batch=0 train binary_f_1.000 <score>=0.756272401434\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:33.932] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 298, \"duration\": 570, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:33 INFO 139813082523456] #quality_metric: host=algo-1, epoch=148, train binary_classification_accuracy <score>=0.75689010989\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:33 INFO 139813082523456] #quality_metric: host=algo-1, epoch=148, train binary_classification_cross_entropy <loss>=0.499464657291\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:33 INFO 139813082523456] #quality_metric: host=algo-1, epoch=148, train binary_f_1.000 <score>=0.785822853436\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 571.6431140899658, \"sum\": 571.6431140899658, \"min\": 571.6431140899658}}, \"EndTime\": 1604871693.932799, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871693.360441}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:33 INFO 139813082523456] #progress_metric: host=algo-1, completed 74 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 13560, \"sum\": 13560.0, \"min\": 13560}, \"Total Records Seen\": {\"count\": 1, \"max\": 13495930, \"sum\": 13495930.0, \"min\": 13495930}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 150, \"sum\": 150.0, \"min\": 150}}, \"EndTime\": 1604871693.932948, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 148}, \"StartTime\": 1604871693.361129}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:33 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=158365.425106 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:33 INFO 139813082523456] #quality_metric: host=algo-1, epoch=149, batch=0 train binary_classification_accuracy <score>=0.728\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:33 INFO 139813082523456] #quality_metric: host=algo-1, epoch=149, batch=0 train binary_classification_cross_entropy <loss>=0.522739196777\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:33 INFO 139813082523456] #quality_metric: host=algo-1, epoch=149, batch=0 train binary_f_1.000 <score>=0.756272401434\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:34.500] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 300, \"duration\": 565, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:34 INFO 139813082523456] #quality_metric: host=algo-1, epoch=149, train binary_classification_accuracy <score>=0.757131868132\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:34 INFO 139813082523456] #quality_metric: host=algo-1, epoch=149, train binary_classification_cross_entropy <loss>=0.499141994602\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:34 INFO 139813082523456] #quality_metric: host=algo-1, epoch=149, train binary_f_1.000 <score>=0.786044125192\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 567.1079158782959, \"sum\": 567.1079158782959, \"min\": 567.1079158782959}}, \"EndTime\": 1604871694.500573, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871693.932855}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:34 INFO 139813082523456] #progress_metric: host=algo-1, completed 75 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 13651, \"sum\": 13651.0, \"min\": 13651}, \"Total Records Seen\": {\"count\": 1, \"max\": 13586500, \"sum\": 13586500.0, \"min\": 13586500}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 151, \"sum\": 151.0, \"min\": 151}}, \"EndTime\": 1604871694.500797, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 149}, \"StartTime\": 1604871693.933435}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:34 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=159602.928931 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:34 INFO 139813082523456] #quality_metric: host=algo-1, epoch=150, batch=0 train binary_classification_accuracy <score>=0.729\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:34 INFO 139813082523456] #quality_metric: host=algo-1, epoch=150, batch=0 train binary_classification_cross_entropy <loss>=0.522395019531\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:34 INFO 139813082523456] #quality_metric: host=algo-1, epoch=150, batch=0 train binary_f_1.000 <score>=0.756950672646\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:35.113] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 302, \"duration\": 610, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:35 INFO 139813082523456] #quality_metric: host=algo-1, epoch=150, train binary_classification_accuracy <score>=0.757406593407\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:35 INFO 139813082523456] #quality_metric: host=algo-1, epoch=150, train binary_classification_cross_entropy <loss>=0.49881816252\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:35 INFO 139813082523456] #quality_metric: host=algo-1, epoch=150, train binary_f_1.000 <score>=0.786308901537\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 612.5741004943848, \"sum\": 612.5741004943848, \"min\": 612.5741004943848}}, \"EndTime\": 1604871695.113936, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871694.500636}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:35 INFO 139813082523456] #progress_metric: host=algo-1, completed 75 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 13742, \"sum\": 13742.0, \"min\": 13742}, \"Total Records Seen\": {\"count\": 1, \"max\": 13677070, \"sum\": 13677070.0, \"min\": 13677070}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 152, \"sum\": 152.0, \"min\": 152}}, \"EndTime\": 1604871695.114189, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 150}, \"StartTime\": 1604871694.501337}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:35 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=147762.762059 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:35 INFO 139813082523456] #quality_metric: host=algo-1, epoch=151, batch=0 train binary_classification_accuracy <score>=0.73\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:35 INFO 139813082523456] #quality_metric: host=algo-1, epoch=151, batch=0 train binary_classification_cross_entropy <loss>=0.522048706055\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:35 INFO 139813082523456] #quality_metric: host=algo-1, epoch=151, batch=0 train binary_f_1.000 <score>=0.75763016158\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:35.658] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 304, \"duration\": 542, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:35 INFO 139813082523456] #quality_metric: host=algo-1, epoch=151, train binary_classification_accuracy <score>=0.757626373626\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:35 INFO 139813082523456] #quality_metric: host=algo-1, epoch=151, train binary_classification_cross_entropy <loss>=0.498493131533\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:35 INFO 139813082523456] #quality_metric: host=algo-1, epoch=151, train binary_f_1.000 <score>=0.78649423062\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 544.0471172332764, \"sum\": 544.0471172332764, \"min\": 544.0471172332764}}, \"EndTime\": 1604871695.658747, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871695.113991}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:35 INFO 139813082523456] #progress_metric: host=algo-1, completed 76 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 13833, \"sum\": 13833.0, \"min\": 13833}, \"Total Records Seen\": {\"count\": 1, \"max\": 13767640, \"sum\": 13767640.0, \"min\": 13767640}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 153, \"sum\": 153.0, \"min\": 153}}, \"EndTime\": 1604871695.658895, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 151}, \"StartTime\": 1604871695.114675}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:35 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=166394.121985 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:35 INFO 139813082523456] #quality_metric: host=algo-1, epoch=152, batch=0 train binary_classification_accuracy <score>=0.73\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:35 INFO 139813082523456] #quality_metric: host=algo-1, epoch=152, batch=0 train binary_classification_cross_entropy <loss>=0.521700073242\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:35 INFO 139813082523456] #quality_metric: host=algo-1, epoch=152, batch=0 train binary_f_1.000 <score>=0.75763016158\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:36.235] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 306, \"duration\": 574, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:36 INFO 139813082523456] #quality_metric: host=algo-1, epoch=152, train binary_classification_accuracy <score>=0.757868131868\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:36 INFO 139813082523456] #quality_metric: host=algo-1, epoch=152, train binary_classification_cross_entropy <loss>=0.498166868105\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:36 INFO 139813082523456] #quality_metric: host=algo-1, epoch=152, train binary_f_1.000 <score>=0.786694805321\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 576.2691497802734, \"sum\": 576.2691497802734, \"min\": 576.2691497802734}}, \"EndTime\": 1604871696.235675, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871695.658797}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:36 INFO 139813082523456] #progress_metric: host=algo-1, completed 76 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 13924, \"sum\": 13924.0, \"min\": 13924}, \"Total Records Seen\": {\"count\": 1, \"max\": 13858210, \"sum\": 13858210.0, \"min\": 13858210}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 154, \"sum\": 154.0, \"min\": 154}}, \"EndTime\": 1604871696.235825, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 152}, \"StartTime\": 1604871695.659378}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:36 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=157094.19507 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:36 INFO 139813082523456] #quality_metric: host=algo-1, epoch=153, batch=0 train binary_classification_accuracy <score>=0.731\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:36 INFO 139813082523456] #quality_metric: host=algo-1, epoch=153, batch=0 train binary_classification_cross_entropy <loss>=0.521349243164\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:36 INFO 139813082523456] #quality_metric: host=algo-1, epoch=153, batch=0 train binary_f_1.000 <score>=0.758744394619\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:36.790] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 308, \"duration\": 552, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:36 INFO 139813082523456] #quality_metric: host=algo-1, epoch=153, train binary_classification_accuracy <score>=0.758164835165\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:36 INFO 139813082523456] #quality_metric: host=algo-1, epoch=153, train binary_classification_cross_entropy <loss>=0.497839349768\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:36 INFO 139813082523456] #quality_metric: host=algo-1, epoch=153, train binary_f_1.000 <score>=0.786970620977\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 554.542064666748, \"sum\": 554.542064666748, \"min\": 554.542064666748}}, \"EndTime\": 1604871696.790893, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871696.23573}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:36 INFO 139813082523456] #progress_metric: host=algo-1, completed 77 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 14015, \"sum\": 14015.0, \"min\": 14015}, \"Total Records Seen\": {\"count\": 1, \"max\": 13948780, \"sum\": 13948780.0, \"min\": 13948780}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 155, \"sum\": 155.0, \"min\": 155}}, \"EndTime\": 1604871696.791115, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 153}, \"StartTime\": 1604871696.23632}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:36 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=163215.392403 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:36 INFO 139813082523456] #quality_metric: host=algo-1, epoch=154, batch=0 train binary_classification_accuracy <score>=0.731\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:36 INFO 139813082523456] #quality_metric: host=algo-1, epoch=154, batch=0 train binary_classification_cross_entropy <loss>=0.52099621582\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:36 INFO 139813082523456] #quality_metric: host=algo-1, epoch=154, batch=0 train binary_f_1.000 <score>=0.758744394619\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:37.345] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 310, \"duration\": 552, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:37 INFO 139813082523456] #quality_metric: host=algo-1, epoch=154, train binary_classification_accuracy <score>=0.758472527473\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:37 INFO 139813082523456] #quality_metric: host=algo-1, epoch=154, train binary_classification_cross_entropy <loss>=0.497510548351\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:37 INFO 139813082523456] #quality_metric: host=algo-1, epoch=154, train binary_f_1.000 <score>=0.787229305221\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 553.6940097808838, \"sum\": 553.6940097808838, \"min\": 553.6940097808838}}, \"EndTime\": 1604871697.345391, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871696.790958}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:37 INFO 139813082523456] #progress_metric: host=algo-1, completed 77 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 14106, \"sum\": 14106.0, \"min\": 14106}, \"Total Records Seen\": {\"count\": 1, \"max\": 14039350, \"sum\": 14039350.0, \"min\": 14039350}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 156, \"sum\": 156.0, \"min\": 156}}, \"EndTime\": 1604871697.345543, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 154}, \"StartTime\": 1604871696.791672}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:37 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=163496.09712 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:37 INFO 139813082523456] #quality_metric: host=algo-1, epoch=155, batch=0 train binary_classification_accuracy <score>=0.731\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:37 INFO 139813082523456] #quality_metric: host=algo-1, epoch=155, batch=0 train binary_classification_cross_entropy <loss>=0.52064074707\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:37 INFO 139813082523456] #quality_metric: host=algo-1, epoch=155, batch=0 train binary_f_1.000 <score>=0.758744394619\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:37.909] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 312, \"duration\": 562, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:37 INFO 139813082523456] #quality_metric: host=algo-1, epoch=155, train binary_classification_accuracy <score>=0.758648351648\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:37 INFO 139813082523456] #quality_metric: host=algo-1, epoch=155, train binary_classification_cross_entropy <loss>=0.497180439373\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:37 INFO 139813082523456] #quality_metric: host=algo-1, epoch=155, train binary_f_1.000 <score>=0.787408891599\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 563.8210773468018, \"sum\": 563.8210773468018, \"min\": 563.8210773468018}}, \"EndTime\": 1604871697.909918, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871697.345447}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:37 INFO 139813082523456] #progress_metric: host=algo-1, completed 78 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 14197, \"sum\": 14197.0, \"min\": 14197}, \"Total Records Seen\": {\"count\": 1, \"max\": 14129920, \"sum\": 14129920.0, \"min\": 14129920}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 157, \"sum\": 157.0, \"min\": 157}}, \"EndTime\": 1604871697.910158, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 155}, \"StartTime\": 1604871697.346069}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:37 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=160529.356202 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:37 INFO 139813082523456] #quality_metric: host=algo-1, epoch=156, batch=0 train binary_classification_accuracy <score>=0.731\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:37 INFO 139813082523456] #quality_metric: host=algo-1, epoch=156, batch=0 train binary_classification_cross_entropy <loss>=0.520283081055\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:37 INFO 139813082523456] #quality_metric: host=algo-1, epoch=156, batch=0 train binary_f_1.000 <score>=0.758744394619\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[34m[2020-11-08 21:41:38.472] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 314, \"duration\": 560, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:38 INFO 139813082523456] #quality_metric: host=algo-1, epoch=156, train binary_classification_accuracy <score>=0.75878021978\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:38 INFO 139813082523456] #quality_metric: host=algo-1, epoch=156, train binary_classification_cross_entropy <loss>=0.496849003048\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:38 INFO 139813082523456] #quality_metric: host=algo-1, epoch=156, train binary_f_1.000 <score>=0.787516818802\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 562.0059967041016, \"sum\": 562.0059967041016, \"min\": 562.0059967041016}}, \"EndTime\": 1604871698.472673, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871697.909978}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:38 INFO 139813082523456] #progress_metric: host=algo-1, completed 78 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 14288, \"sum\": 14288.0, \"min\": 14288}, \"Total Records Seen\": {\"count\": 1, \"max\": 14220490, \"sum\": 14220490.0, \"min\": 14220490}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 158, \"sum\": 158.0, \"min\": 158}}, \"EndTime\": 1604871698.472946, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 156}, \"StartTime\": 1604871697.910639}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:38 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=161037.427899 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:38 INFO 139813082523456] #quality_metric: host=algo-1, epoch=157, batch=0 train binary_classification_accuracy <score>=0.731\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:38 INFO 139813082523456] #quality_metric: host=algo-1, epoch=157, batch=0 train binary_classification_cross_entropy <loss>=0.519923095703\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:38 INFO 139813082523456] #quality_metric: host=algo-1, epoch=157, batch=0 train binary_f_1.000 <score>=0.758744394619\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:39.004] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 316, \"duration\": 529, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:39 INFO 139813082523456] #quality_metric: host=algo-1, epoch=157, train binary_classification_accuracy <score>=0.758945054945\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:39 INFO 139813082523456] #quality_metric: host=algo-1, epoch=157, train binary_classification_cross_entropy <loss>=0.496516207852\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:39 INFO 139813082523456] #quality_metric: host=algo-1, epoch=157, train binary_f_1.000 <score>=0.787639405204\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 531.3389301300049, \"sum\": 531.3389301300049, \"min\": 531.3389301300049}}, \"EndTime\": 1604871699.004835, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871698.47277}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:39 INFO 139813082523456] #progress_metric: host=algo-1, completed 79 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 14379, \"sum\": 14379.0, \"min\": 14379}, \"Total Records Seen\": {\"count\": 1, \"max\": 14311060, \"sum\": 14311060.0, \"min\": 14311060}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 159, \"sum\": 159.0, \"min\": 159}}, \"EndTime\": 1604871699.005015, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 157}, \"StartTime\": 1604871698.473466}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:39 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=170359.624945 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:39 INFO 139813082523456] #quality_metric: host=algo-1, epoch=158, batch=0 train binary_classification_accuracy <score>=0.731\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:39 INFO 139813082523456] #quality_metric: host=algo-1, epoch=158, batch=0 train binary_classification_cross_entropy <loss>=0.519560668945\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:39 INFO 139813082523456] #quality_metric: host=algo-1, epoch=158, batch=0 train binary_f_1.000 <score>=0.758744394619\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:39.552] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 318, \"duration\": 545, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:39 INFO 139813082523456] #quality_metric: host=algo-1, epoch=158, train binary_classification_accuracy <score>=0.759197802198\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:39 INFO 139813082523456] #quality_metric: host=algo-1, epoch=158, train binary_classification_cross_entropy <loss>=0.496182034335\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:39 INFO 139813082523456] #quality_metric: host=algo-1, epoch=158, train binary_f_1.000 <score>=0.787876441149\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 546.9050407409668, \"sum\": 546.9050407409668, \"min\": 546.9050407409668}}, \"EndTime\": 1604871699.552404, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871699.004887}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:39 INFO 139813082523456] #progress_metric: host=algo-1, completed 79 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 14470, \"sum\": 14470.0, \"min\": 14470}, \"Total Records Seen\": {\"count\": 1, \"max\": 14401630, \"sum\": 14401630.0, \"min\": 14401630}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 160, \"sum\": 160.0, \"min\": 160}}, \"EndTime\": 1604871699.552555, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 158}, \"StartTime\": 1604871699.005473}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:39 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=165524.088118 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:39 INFO 139813082523456] #quality_metric: host=algo-1, epoch=159, batch=0 train binary_classification_accuracy <score>=0.731\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:39 INFO 139813082523456] #quality_metric: host=algo-1, epoch=159, batch=0 train binary_classification_cross_entropy <loss>=0.519195922852\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:39 INFO 139813082523456] #quality_metric: host=algo-1, epoch=159, batch=0 train binary_f_1.000 <score>=0.758744394619\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:40.128] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 320, \"duration\": 573, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:40 INFO 139813082523456] #quality_metric: host=algo-1, epoch=159, train binary_classification_accuracy <score>=0.759428571429\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:40 INFO 139813082523456] #quality_metric: host=algo-1, epoch=159, train binary_classification_cross_entropy <loss>=0.495846454998\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:40 INFO 139813082523456] #quality_metric: host=algo-1, epoch=159, train binary_f_1.000 <score>=0.788077675166\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 575.70481300354, \"sum\": 575.70481300354, \"min\": 575.70481300354}}, \"EndTime\": 1604871700.128792, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871699.55246}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:40 INFO 139813082523456] #progress_metric: host=algo-1, completed 80 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 14561, \"sum\": 14561.0, \"min\": 14561}, \"Total Records Seen\": {\"count\": 1, \"max\": 14492200, \"sum\": 14492200.0, \"min\": 14492200}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 161, \"sum\": 161.0, \"min\": 161}}, \"EndTime\": 1604871700.129148, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 159}, \"StartTime\": 1604871699.553058}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:40 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=157187.734628 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:40 INFO 139813082523456] #quality_metric: host=algo-1, epoch=160, batch=0 train binary_classification_accuracy <score>=0.731\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:40 INFO 139813082523456] #quality_metric: host=algo-1, epoch=160, batch=0 train binary_classification_cross_entropy <loss>=0.518828735352\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:40 INFO 139813082523456] #quality_metric: host=algo-1, epoch=160, batch=0 train binary_f_1.000 <score>=0.758744394619\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:40.698] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 322, \"duration\": 566, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:40 INFO 139813082523456] #quality_metric: host=algo-1, epoch=160, train binary_classification_accuracy <score>=0.759692307692\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:40 INFO 139813082523456] #quality_metric: host=algo-1, epoch=160, train binary_classification_cross_entropy <loss>=0.495509451395\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:40 INFO 139813082523456] #quality_metric: host=algo-1, epoch=160, train binary_f_1.000 <score>=0.788318200298\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 568.8300132751465, \"sum\": 568.8300132751465, \"min\": 568.8300132751465}}, \"EndTime\": 1604871700.69865, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871700.128891}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:40 INFO 139813082523456] #progress_metric: host=algo-1, completed 80 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 14652, \"sum\": 14652.0, \"min\": 14652}, \"Total Records Seen\": {\"count\": 1, \"max\": 14582770, \"sum\": 14582770.0, \"min\": 14582770}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 162, \"sum\": 162.0, \"min\": 162}}, \"EndTime\": 1604871700.698798, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 160}, \"StartTime\": 1604871700.129792}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:40 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=159145.180388 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:40 INFO 139813082523456] #quality_metric: host=algo-1, epoch=161, batch=0 train binary_classification_accuracy <score>=0.731\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:40 INFO 139813082523456] #quality_metric: host=algo-1, epoch=161, batch=0 train binary_classification_cross_entropy <loss>=0.518459228516\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:40 INFO 139813082523456] #quality_metric: host=algo-1, epoch=161, batch=0 train binary_f_1.000 <score>=0.758744394619\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:41.238] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 324, \"duration\": 537, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:41 INFO 139813082523456] #quality_metric: host=algo-1, epoch=161, train binary_classification_accuracy <score>=0.759813186813\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:41 INFO 139813082523456] #quality_metric: host=algo-1, epoch=161, train binary_classification_cross_entropy <loss>=0.4951710081\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:41 INFO 139813082523456] #quality_metric: host=algo-1, epoch=161, train binary_f_1.000 <score>=0.788410342791\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 539.63303565979, \"sum\": 539.63303565979, \"min\": 539.63303565979}}, \"EndTime\": 1604871701.238959, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871700.698706}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:41 INFO 139813082523456] #progress_metric: host=algo-1, completed 81 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 14743, \"sum\": 14743.0, \"min\": 14743}, \"Total Records Seen\": {\"count\": 1, \"max\": 14673340, \"sum\": 14673340.0, \"min\": 14673340}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 163, \"sum\": 163.0, \"min\": 163}}, \"EndTime\": 1604871701.239159, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 161}, \"StartTime\": 1604871700.699299}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:41 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=167735.564496 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:41 INFO 139813082523456] #quality_metric: host=algo-1, epoch=162, batch=0 train binary_classification_accuracy <score>=0.732\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:41 INFO 139813082523456] #quality_metric: host=algo-1, epoch=162, batch=0 train binary_classification_cross_entropy <loss>=0.518087341309\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:41 INFO 139813082523456] #quality_metric: host=algo-1, epoch=162, batch=0 train binary_f_1.000 <score>=0.759856630824\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:41.789] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 326, \"duration\": 548, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:41 INFO 139813082523456] #quality_metric: host=algo-1, epoch=162, train binary_classification_accuracy <score>=0.760032967033\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:41 INFO 139813082523456] #quality_metric: host=algo-1, epoch=162, train binary_classification_cross_entropy <loss>=0.494831089229\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:41 INFO 139813082523456] #quality_metric: host=algo-1, epoch=162, train binary_f_1.000 <score>=0.788608048325\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 550.2631664276123, \"sum\": 550.2631664276123, \"min\": 550.2631664276123}}, \"EndTime\": 1604871701.789923, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871701.239024}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:41 INFO 139813082523456] #progress_metric: host=algo-1, completed 81 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 14834, \"sum\": 14834.0, \"min\": 14834}, \"Total Records Seen\": {\"count\": 1, \"max\": 14763910, \"sum\": 14763910.0, \"min\": 14763910}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 164, \"sum\": 164.0, \"min\": 164}}, \"EndTime\": 1604871701.790155, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 162}, \"StartTime\": 1604871701.239636}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:41 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=164488.492408 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:41 INFO 139813082523456] #quality_metric: host=algo-1, epoch=163, batch=0 train binary_classification_accuracy <score>=0.733\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:41 INFO 139813082523456] #quality_metric: host=algo-1, epoch=163, batch=0 train binary_classification_cross_entropy <loss>=0.51771295166\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:41 INFO 139813082523456] #quality_metric: host=algo-1, epoch=163, batch=0 train binary_f_1.000 <score>=0.76096687556\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:42.350] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 328, \"duration\": 559, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:42 INFO 139813082523456] #quality_metric: host=algo-1, epoch=163, train binary_classification_accuracy <score>=0.760241758242\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:42 INFO 139813082523456] #quality_metric: host=algo-1, epoch=163, train binary_classification_cross_entropy <loss>=0.494489680699\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:42 INFO 139813082523456] #quality_metric: host=algo-1, epoch=163, train binary_f_1.000 <score>=0.788785842901\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 560.6040954589844, \"sum\": 560.6040954589844, \"min\": 560.6040954589844}}, \"EndTime\": 1604871702.351277, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871701.789981}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:42 INFO 139813082523456] #progress_metric: host=algo-1, completed 82 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 14925, \"sum\": 14925.0, \"min\": 14925}, \"Total Records Seen\": {\"count\": 1, \"max\": 14854480, \"sum\": 14854480.0, \"min\": 14854480}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 165, \"sum\": 165.0, \"min\": 165}}, \"EndTime\": 1604871702.351422, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 163}, \"StartTime\": 1604871701.790649}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:42 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=161484.721951 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:42 INFO 139813082523456] #quality_metric: host=algo-1, epoch=164, batch=0 train binary_classification_accuracy <score>=0.733\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:42 INFO 139813082523456] #quality_metric: host=algo-1, epoch=164, batch=0 train binary_classification_cross_entropy <loss>=0.517336120605\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:42 INFO 139813082523456] #quality_metric: host=algo-1, epoch=164, batch=0 train binary_f_1.000 <score>=0.76096687556\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:42.927] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 330, \"duration\": 574, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:42 INFO 139813082523456] #quality_metric: host=algo-1, epoch=164, train binary_classification_accuracy <score>=0.760516483516\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:42 INFO 139813082523456] #quality_metric: host=algo-1, epoch=164, train binary_classification_cross_entropy <loss>=0.49414676574\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:42 INFO 139813082523456] #quality_metric: host=algo-1, epoch=164, train binary_f_1.000 <score>=0.789046240816\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 576.4379501342773, \"sum\": 576.4379501342773, \"min\": 576.4379501342773}}, \"EndTime\": 1604871702.928388, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871702.351331}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:42 INFO 139813082523456] #progress_metric: host=algo-1, completed 82 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 15016, \"sum\": 15016.0, \"min\": 15016}, \"Total Records Seen\": {\"count\": 1, \"max\": 14945050, \"sum\": 14945050.0, \"min\": 14945050}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 166, \"sum\": 166.0, \"min\": 166}}, \"EndTime\": 1604871702.928614, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 164}, \"StartTime\": 1604871702.351921}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:42 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=157023.934935 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:42 INFO 139813082523456] #quality_metric: host=algo-1, epoch=165, batch=0 train binary_classification_accuracy <score>=0.732\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:42 INFO 139813082523456] #quality_metric: host=algo-1, epoch=165, batch=0 train binary_classification_cross_entropy <loss>=0.516956848145\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:42 INFO 139813082523456] #quality_metric: host=algo-1, epoch=165, batch=0 train binary_f_1.000 <score>=0.760286225403\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:43.499] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 332, \"duration\": 569, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:43 INFO 139813082523456] #quality_metric: host=algo-1, epoch=165, train binary_classification_accuracy <score>=0.760703296703\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:43 INFO 139813082523456] #quality_metric: host=algo-1, epoch=165, train binary_classification_cross_entropy <loss>=0.49380231719\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:43 INFO 139813082523456] #quality_metric: host=algo-1, epoch=165, train binary_f_1.000 <score>=0.789225080822\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 570.9800720214844, \"sum\": 570.9800720214844, \"min\": 570.9800720214844}}, \"EndTime\": 1604871703.500172, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871702.928455}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:43 INFO 139813082523456] #progress_metric: host=algo-1, completed 83 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 15107, \"sum\": 15107.0, \"min\": 15107}, \"Total Records Seen\": {\"count\": 1, \"max\": 15035620, \"sum\": 15035620.0, \"min\": 15035620}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 167, \"sum\": 167.0, \"min\": 167}}, \"EndTime\": 1604871703.50038, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 165}, \"StartTime\": 1604871702.92913}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:43 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=158518.475531 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:43 INFO 139813082523456] #quality_metric: host=algo-1, epoch=166, batch=0 train binary_classification_accuracy <score>=0.731\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:43 INFO 139813082523456] #quality_metric: host=algo-1, epoch=166, batch=0 train binary_classification_cross_entropy <loss>=0.516575073242\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:43 INFO 139813082523456] #quality_metric: host=algo-1, epoch=166, batch=0 train binary_f_1.000 <score>=0.759606791778\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:44.040] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 334, \"duration\": 538, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:44 INFO 139813082523456] #quality_metric: host=algo-1, epoch=166, train binary_classification_accuracy <score>=0.760956043956\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:44 INFO 139813082523456] #quality_metric: host=algo-1, epoch=166, train binary_classification_cross_entropy <loss>=0.493456320291\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:44 INFO 139813082523456] #quality_metric: host=algo-1, epoch=166, train binary_f_1.000 <score>=0.789461968041\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 539.7489070892334, \"sum\": 539.7489070892334, \"min\": 539.7489070892334}}, \"EndTime\": 1604871704.040653, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871703.50023}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:44 INFO 139813082523456] #progress_metric: host=algo-1, completed 83 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 15198, \"sum\": 15198.0, \"min\": 15198}, \"Total Records Seen\": {\"count\": 1, \"max\": 15126190, \"sum\": 15126190.0, \"min\": 15126190}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 168, \"sum\": 168.0, \"min\": 168}}, \"EndTime\": 1604871704.040804, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 166}, \"StartTime\": 1604871703.500878}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:44 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=167718.457502 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:44 INFO 139813082523456] #quality_metric: host=algo-1, epoch=167, batch=0 train binary_classification_accuracy <score>=0.731\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:44 INFO 139813082523456] #quality_metric: host=algo-1, epoch=167, batch=0 train binary_classification_cross_entropy <loss>=0.516190795898\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:44 INFO 139813082523456] #quality_metric: host=algo-1, epoch=167, batch=0 train binary_f_1.000 <score>=0.759606791778\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:44.583] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 336, \"duration\": 541, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:44 INFO 139813082523456] #quality_metric: host=algo-1, epoch=167, train binary_classification_accuracy <score>=0.76132967033\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:44 INFO 139813082523456] #quality_metric: host=algo-1, epoch=167, train binary_classification_cross_entropy <loss>=0.49310875224\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:44 INFO 139813082523456] #quality_metric: host=algo-1, epoch=167, train binary_f_1.000 <score>=0.789786970451\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 542.9160594940186, \"sum\": 542.9160594940186, \"min\": 542.9160594940186}}, \"EndTime\": 1604871704.584253, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871704.040709}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:44 INFO 139813082523456] #progress_metric: host=algo-1, completed 84 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 15289, \"sum\": 15289.0, \"min\": 15289}, \"Total Records Seen\": {\"count\": 1, \"max\": 15216760, \"sum\": 15216760.0, \"min\": 15216760}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 169, \"sum\": 169.0, \"min\": 169}}, \"EndTime\": 1604871704.584461, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 167}, \"StartTime\": 1604871704.041308}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:44 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=166715.210965 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:44 INFO 139813082523456] #quality_metric: host=algo-1, epoch=168, batch=0 train binary_classification_accuracy <score>=0.732\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:44 INFO 139813082523456] #quality_metric: host=algo-1, epoch=168, batch=0 train binary_classification_cross_entropy <loss>=0.515804077148\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:44 INFO 139813082523456] #quality_metric: host=algo-1, epoch=168, batch=0 train binary_f_1.000 <score>=0.760286225403\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:45.170] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 338, \"duration\": 583, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:45 INFO 139813082523456] #quality_metric: host=algo-1, epoch=168, train binary_classification_accuracy <score>=0.761538461538\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:45 INFO 139813082523456] #quality_metric: host=algo-1, epoch=168, train binary_classification_cross_entropy <loss>=0.492759595263\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:45 INFO 139813082523456] #quality_metric: host=algo-1, epoch=168, train binary_f_1.000 <score>=0.789948503504\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 585.7179164886475, \"sum\": 585.7179164886475, \"min\": 585.7179164886475}}, \"EndTime\": 1604871705.170709, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871704.584321}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:45 INFO 139813082523456] #progress_metric: host=algo-1, completed 84 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 15380, \"sum\": 15380.0, \"min\": 15380}, \"Total Records Seen\": {\"count\": 1, \"max\": 15307330, \"sum\": 15307330.0, \"min\": 15307330}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 170, \"sum\": 170.0, \"min\": 170}}, \"EndTime\": 1604871705.170888, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 168}, \"StartTime\": 1604871704.584963}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:45 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=154549.469168 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:45 INFO 139813082523456] #quality_metric: host=algo-1, epoch=169, batch=0 train binary_classification_accuracy <score>=0.732\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:45 INFO 139813082523456] #quality_metric: host=algo-1, epoch=169, batch=0 train binary_classification_cross_entropy <loss>=0.515414794922\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:45 INFO 139813082523456] #quality_metric: host=algo-1, epoch=169, batch=0 train binary_f_1.000 <score>=0.760286225403\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:45.741] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 340, \"duration\": 568, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:45 INFO 139813082523456] #quality_metric: host=algo-1, epoch=169, train binary_classification_accuracy <score>=0.761714285714\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:45 INFO 139813082523456] #quality_metric: host=algo-1, epoch=169, train binary_classification_cross_entropy <loss>=0.492408833933\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:45 INFO 139813082523456] #quality_metric: host=algo-1, epoch=169, train binary_f_1.000 <score>=0.790107443616\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 570.5368518829346, \"sum\": 570.5368518829346, \"min\": 570.5368518829346}}, \"EndTime\": 1604871705.741962, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871705.170765}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:45 INFO 139813082523456] #progress_metric: host=algo-1, completed 85 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 15471, \"sum\": 15471.0, \"min\": 15471}, \"Total Records Seen\": {\"count\": 1, \"max\": 15397900, \"sum\": 15397900.0, \"min\": 15397900}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 171, \"sum\": 171.0, \"min\": 171}}, \"EndTime\": 1604871705.742164, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 169}, \"StartTime\": 1604871705.1714}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:45 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=158655.05605 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:45 INFO 139813082523456] #quality_metric: host=algo-1, epoch=170, batch=0 train binary_classification_accuracy <score>=0.732\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:45 INFO 139813082523456] #quality_metric: host=algo-1, epoch=170, batch=0 train binary_classification_cross_entropy <loss>=0.515022949219\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:45 INFO 139813082523456] #quality_metric: host=algo-1, epoch=170, batch=0 train binary_f_1.000 <score>=0.760286225403\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:46.320] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 342, \"duration\": 576, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:46 INFO 139813082523456] #quality_metric: host=algo-1, epoch=170, train binary_classification_accuracy <score>=0.761934065934\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:46 INFO 139813082523456] #quality_metric: host=algo-1, epoch=170, train binary_classification_cross_entropy <loss>=0.492056442093\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:46 INFO 139813082523456] #quality_metric: host=algo-1, epoch=170, train binary_f_1.000 <score>=0.790305095245\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 577.7881145477295, \"sum\": 577.7881145477295, \"min\": 577.7881145477295}}, \"EndTime\": 1604871706.32046, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871705.742039}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:46 INFO 139813082523456] #progress_metric: host=algo-1, completed 85 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 15562, \"sum\": 15562.0, \"min\": 15562}, \"Total Records Seen\": {\"count\": 1, \"max\": 15488470, \"sum\": 15488470.0, \"min\": 15488470}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 172, \"sum\": 172.0, \"min\": 172}}, \"EndTime\": 1604871706.320629, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 170}, \"StartTime\": 1604871705.742647}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:46 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=156676.224211 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:46 INFO 139813082523456] #quality_metric: host=algo-1, epoch=171, batch=0 train binary_classification_accuracy <score>=0.733\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:46 INFO 139813082523456] #quality_metric: host=algo-1, epoch=171, batch=0 train binary_classification_cross_entropy <loss>=0.514628540039\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:46 INFO 139813082523456] #quality_metric: host=algo-1, epoch=171, batch=0 train binary_f_1.000 <score>=0.76096687556\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:46.868] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 344, \"duration\": 546, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:46 INFO 139813082523456] #quality_metric: host=algo-1, epoch=171, train binary_classification_accuracy <score>=0.762208791209\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:46 INFO 139813082523456] #quality_metric: host=algo-1, epoch=171, train binary_classification_cross_entropy <loss>=0.491702406663\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:46 INFO 139813082523456] #quality_metric: host=algo-1, epoch=171, train binary_f_1.000 <score>=0.790557217109\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 547.9400157928467, \"sum\": 547.9400157928467, \"min\": 547.9400157928467}}, \"EndTime\": 1604871706.869047, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871706.32052}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:46 INFO 139813082523456] #progress_metric: host=algo-1, completed 86 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 15653, \"sum\": 15653.0, \"min\": 15653}, \"Total Records Seen\": {\"count\": 1, \"max\": 15579040, \"sum\": 15579040.0, \"min\": 15579040}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 173, \"sum\": 173.0, \"min\": 173}}, \"EndTime\": 1604871706.869197, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 171}, \"StartTime\": 1604871706.321082}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:46 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=165213.099027 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:46 INFO 139813082523456] #quality_metric: host=algo-1, epoch=172, batch=0 train binary_classification_accuracy <score>=0.733\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:46 INFO 139813082523456] #quality_metric: host=algo-1, epoch=172, batch=0 train binary_classification_cross_entropy <loss>=0.514231628418\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:46 INFO 139813082523456] #quality_metric: host=algo-1, epoch=172, batch=0 train binary_f_1.000 <score>=0.76096687556\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:47.417] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 346, \"duration\": 546, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:47 INFO 139813082523456] #quality_metric: host=algo-1, epoch=172, train binary_classification_accuracy <score>=0.76243956044\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:47 INFO 139813082523456] #quality_metric: host=algo-1, epoch=172, train binary_classification_cross_entropy <loss>=0.491346707858\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:47 INFO 139813082523456] #quality_metric: host=algo-1, epoch=172, train binary_f_1.000 <score>=0.79075439921\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 547.9950904846191, \"sum\": 547.9950904846191, \"min\": 547.9950904846191}}, \"EndTime\": 1604871707.417702, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871706.869103}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:47 INFO 139813082523456] #progress_metric: host=algo-1, completed 86 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 15744, \"sum\": 15744.0, \"min\": 15744}, \"Total Records Seen\": {\"count\": 1, \"max\": 15669610, \"sum\": 15669610.0, \"min\": 15669610}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 174, \"sum\": 174.0, \"min\": 174}}, \"EndTime\": 1604871707.417851, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 172}, \"StartTime\": 1604871706.869683}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:47 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=165198.657843 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:47 INFO 139813082523456] #quality_metric: host=algo-1, epoch=173, batch=0 train binary_classification_accuracy <score>=0.733\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:47 INFO 139813082523456] #quality_metric: host=algo-1, epoch=173, batch=0 train binary_classification_cross_entropy <loss>=0.51383215332\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:47 INFO 139813082523456] #quality_metric: host=algo-1, epoch=173, batch=0 train binary_f_1.000 <score>=0.76096687556\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:47.983] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 348, \"duration\": 563, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:47 INFO 139813082523456] #quality_metric: host=algo-1, epoch=173, train binary_classification_accuracy <score>=0.762714285714\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:47 INFO 139813082523456] #quality_metric: host=algo-1, epoch=173, train binary_classification_cross_entropy <loss>=0.49098933025\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:47 INFO 139813082523456] #quality_metric: host=algo-1, epoch=173, train binary_f_1.000 <score>=0.790986264507\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 565.4511451721191, \"sum\": 565.4511451721191, \"min\": 565.4511451721191}}, \"EndTime\": 1604871707.983849, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871707.417758}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:47 INFO 139813082523456] #progress_metric: host=algo-1, completed 87 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 15835, \"sum\": 15835.0, \"min\": 15835}, \"Total Records Seen\": {\"count\": 1, \"max\": 15760180, \"sum\": 15760180.0, \"min\": 15760180}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 175, \"sum\": 175.0, \"min\": 175}}, \"EndTime\": 1604871707.984059, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 173}, \"StartTime\": 1604871707.418368}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:47 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=160071.074674 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:47 INFO 139813082523456] #quality_metric: host=algo-1, epoch=174, batch=0 train binary_classification_accuracy <score>=0.733\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:47 INFO 139813082523456] #quality_metric: host=algo-1, epoch=174, batch=0 train binary_classification_cross_entropy <loss>=0.513429992676\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:47 INFO 139813082523456] #quality_metric: host=algo-1, epoch=174, batch=0 train binary_f_1.000 <score>=0.76096687556\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[34m[2020-11-08 21:41:48.545] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 350, \"duration\": 559, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:48 INFO 139813082523456] #quality_metric: host=algo-1, epoch=174, train binary_classification_accuracy <score>=0.762956043956\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:48 INFO 139813082523456] #quality_metric: host=algo-1, epoch=174, train binary_classification_cross_entropy <loss>=0.490630248353\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:48 INFO 139813082523456] #quality_metric: host=algo-1, epoch=174, train binary_f_1.000 <score>=0.79119921788\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 561.0589981079102, \"sum\": 561.0589981079102, \"min\": 561.0589981079102}}, \"EndTime\": 1604871708.545651, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871707.983914}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:48 INFO 139813082523456] #progress_metric: host=algo-1, completed 87 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 15926, \"sum\": 15926.0, \"min\": 15926}, \"Total Records Seen\": {\"count\": 1, \"max\": 15850750, \"sum\": 15850750.0, \"min\": 15850750}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 176, \"sum\": 176.0, \"min\": 176}}, \"EndTime\": 1604871708.545851, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 174}, \"StartTime\": 1604871707.984567}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:48 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=161331.099998 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:48 INFO 139813082523456] #quality_metric: host=algo-1, epoch=175, batch=0 train binary_classification_accuracy <score>=0.733\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:48 INFO 139813082523456] #quality_metric: host=algo-1, epoch=175, batch=0 train binary_classification_cross_entropy <loss>=0.513025268555\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:48 INFO 139813082523456] #quality_metric: host=algo-1, epoch=175, batch=0 train binary_f_1.000 <score>=0.76096687556\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:49.103] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 352, \"duration\": 555, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:49 INFO 139813082523456] #quality_metric: host=algo-1, epoch=175, train binary_classification_accuracy <score>=0.763208791209\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:49 INFO 139813082523456] #quality_metric: host=algo-1, epoch=175, train binary_classification_cross_entropy <loss>=0.490269455794\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:49 INFO 139813082523456] #quality_metric: host=algo-1, epoch=175, train binary_f_1.000 <score>=0.791411755595\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 557.2590827941895, \"sum\": 557.2590827941895, \"min\": 557.2590827941895}}, \"EndTime\": 1604871709.103676, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871708.545711}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:49 INFO 139813082523456] #progress_metric: host=algo-1, completed 88 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 16017, \"sum\": 16017.0, \"min\": 16017}, \"Total Records Seen\": {\"count\": 1, \"max\": 15941320, \"sum\": 15941320.0, \"min\": 15941320}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 177, \"sum\": 177.0, \"min\": 177}}, \"EndTime\": 1604871709.10383, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 175}, \"StartTime\": 1604871708.54639}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:49 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=162450.068328 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:49 INFO 139813082523456] #quality_metric: host=algo-1, epoch=176, batch=0 train binary_classification_accuracy <score>=0.733\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:49 INFO 139813082523456] #quality_metric: host=algo-1, epoch=176, batch=0 train binary_classification_cross_entropy <loss>=0.512617919922\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:49 INFO 139813082523456] #quality_metric: host=algo-1, epoch=176, batch=0 train binary_f_1.000 <score>=0.76096687556\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:49.655] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 354, \"duration\": 549, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:49 INFO 139813082523456] #quality_metric: host=algo-1, epoch=176, train binary_classification_accuracy <score>=0.763373626374\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:49 INFO 139813082523456] #quality_metric: host=algo-1, epoch=176, train binary_classification_cross_entropy <loss>=0.489906928765\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:49 INFO 139813082523456] #quality_metric: host=algo-1, epoch=176, train binary_f_1.000 <score>=0.791554940321\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 551.4359474182129, \"sum\": 551.4359474182129, \"min\": 551.4359474182129}}, \"EndTime\": 1604871709.655782, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871709.103732}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:49 INFO 139813082523456] #progress_metric: host=algo-1, completed 88 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 16108, \"sum\": 16108.0, \"min\": 16108}, \"Total Records Seen\": {\"count\": 1, \"max\": 16031890, \"sum\": 16031890.0, \"min\": 16031890}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 178, \"sum\": 178.0, \"min\": 178}}, \"EndTime\": 1604871709.655979, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 176}, \"StartTime\": 1604871709.104319}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:49 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=164145.339987 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:49 INFO 139813082523456] #quality_metric: host=algo-1, epoch=177, batch=0 train binary_classification_accuracy <score>=0.735\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:49 INFO 139813082523456] #quality_metric: host=algo-1, epoch=177, batch=0 train binary_classification_cross_entropy <loss>=0.512207946777\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:49 INFO 139813082523456] #quality_metric: host=algo-1, epoch=177, batch=0 train binary_f_1.000 <score>=0.762757385855\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:50.224] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 356, \"duration\": 566, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:50 INFO 139813082523456] #quality_metric: host=algo-1, epoch=177, train binary_classification_accuracy <score>=0.763736263736\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:50 INFO 139813082523456] #quality_metric: host=algo-1, epoch=177, train binary_classification_cross_entropy <loss>=0.489542650831\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:50 INFO 139813082523456] #quality_metric: host=algo-1, epoch=177, train binary_f_1.000 <score>=0.791876403624\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 568.5999393463135, \"sum\": 568.5999393463135, \"min\": 568.5999393463135}}, \"EndTime\": 1604871710.22509, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871709.655837}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:50 INFO 139813082523456] #progress_metric: host=algo-1, completed 89 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 16199, \"sum\": 16199.0, \"min\": 16199}, \"Total Records Seen\": {\"count\": 1, \"max\": 16122460, \"sum\": 16122460.0, \"min\": 16122460}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 179, \"sum\": 179.0, \"min\": 179}}, \"EndTime\": 1604871710.225239, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 177}, \"StartTime\": 1604871709.656463}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:50 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=159213.748612 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:50 INFO 139813082523456] #quality_metric: host=algo-1, epoch=178, batch=0 train binary_classification_accuracy <score>=0.735\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:50 INFO 139813082523456] #quality_metric: host=algo-1, epoch=178, batch=0 train binary_classification_cross_entropy <loss>=0.511795288086\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:50 INFO 139813082523456] #quality_metric: host=algo-1, epoch=178, batch=0 train binary_f_1.000 <score>=0.762757385855\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:50.787] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 358, \"duration\": 560, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:50 INFO 139813082523456] #quality_metric: host=algo-1, epoch=178, train binary_classification_accuracy <score>=0.76410989011\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:50 INFO 139813082523456] #quality_metric: host=algo-1, epoch=178, train binary_classification_cross_entropy <loss>=0.489176613944\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:50 INFO 139813082523456] #quality_metric: host=algo-1, epoch=178, train binary_f_1.000 <score>=0.792189436183\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 561.9649887084961, \"sum\": 561.9649887084961, \"min\": 561.9649887084961}}, \"EndTime\": 1604871710.787723, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871710.225146}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:50 INFO 139813082523456] #progress_metric: host=algo-1, completed 89 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 16290, \"sum\": 16290.0, \"min\": 16290}, \"Total Records Seen\": {\"count\": 1, \"max\": 16213030, \"sum\": 16213030.0, \"min\": 16213030}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 180, \"sum\": 180.0, \"min\": 180}}, \"EndTime\": 1604871710.787868, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 178}, \"StartTime\": 1604871710.225734}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:50 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=161092.947921 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:50 INFO 139813082523456] #quality_metric: host=algo-1, epoch=179, batch=0 train binary_classification_accuracy <score>=0.735\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:50 INFO 139813082523456] #quality_metric: host=algo-1, epoch=179, batch=0 train binary_classification_cross_entropy <loss>=0.511380004883\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:50 INFO 139813082523456] #quality_metric: host=algo-1, epoch=179, batch=0 train binary_f_1.000 <score>=0.762757385855\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:51.338] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 360, \"duration\": 548, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:51 INFO 139813082523456] #quality_metric: host=algo-1, epoch=179, train binary_classification_accuracy <score>=0.764208791209\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:51 INFO 139813082523456] #quality_metric: host=algo-1, epoch=179, train binary_classification_cross_entropy <loss>=0.488808785575\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:51 INFO 139813082523456] #quality_metric: host=algo-1, epoch=179, train binary_f_1.000 <score>=0.79227857537\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 550.3849983215332, \"sum\": 550.3849983215332, \"min\": 550.3849983215332}}, \"EndTime\": 1604871711.338775, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871710.787775}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:51 INFO 139813082523456] #progress_metric: host=algo-1, completed 90 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 16381, \"sum\": 16381.0, \"min\": 16381}, \"Total Records Seen\": {\"count\": 1, \"max\": 16303600, \"sum\": 16303600.0, \"min\": 16303600}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 181, \"sum\": 181.0, \"min\": 181}}, \"EndTime\": 1604871711.338999, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 179}, \"StartTime\": 1604871710.788362}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:51 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=164448.474221 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:51 INFO 139813082523456] #quality_metric: host=algo-1, epoch=180, batch=0 train binary_classification_accuracy <score>=0.737\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:51 INFO 139813082523456] #quality_metric: host=algo-1, epoch=180, batch=0 train binary_classification_cross_entropy <loss>=0.510962005615\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:51 INFO 139813082523456] #quality_metric: host=algo-1, epoch=180, batch=0 train binary_f_1.000 <score>=0.76454789615\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:51.883] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 362, \"duration\": 542, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:51 INFO 139813082523456] #quality_metric: host=algo-1, epoch=180, train binary_classification_accuracy <score>=0.764428571429\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:51 INFO 139813082523456] #quality_metric: host=algo-1, epoch=180, train binary_classification_cross_entropy <loss>=0.488439163711\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:51 INFO 139813082523456] #quality_metric: host=algo-1, epoch=180, train binary_f_1.000 <score>=0.792488262911\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 544.4989204406738, \"sum\": 544.4989204406738, \"min\": 544.4989204406738}}, \"EndTime\": 1604871711.884034, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871711.338839}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:51 INFO 139813082523456] #progress_metric: host=algo-1, completed 90 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 16472, \"sum\": 16472.0, \"min\": 16472}, \"Total Records Seen\": {\"count\": 1, \"max\": 16394170, \"sum\": 16394170.0, \"min\": 16394170}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 182, \"sum\": 182.0, \"min\": 182}}, \"EndTime\": 1604871711.884184, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 180}, \"StartTime\": 1604871711.339511}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:51 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=166257.940611 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:51 INFO 139813082523456] #quality_metric: host=algo-1, epoch=181, batch=0 train binary_classification_accuracy <score>=0.737\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:51 INFO 139813082523456] #quality_metric: host=algo-1, epoch=181, batch=0 train binary_classification_cross_entropy <loss>=0.510541320801\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:51 INFO 139813082523456] #quality_metric: host=algo-1, epoch=181, batch=0 train binary_f_1.000 <score>=0.76454789615\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:52.436] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 364, \"duration\": 550, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:52 INFO 139813082523456] #quality_metric: host=algo-1, epoch=181, train binary_classification_accuracy <score>=0.764604395604\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:52 INFO 139813082523456] #quality_metric: host=algo-1, epoch=181, train binary_classification_cross_entropy <loss>=0.488067724542\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:52 INFO 139813082523456] #quality_metric: host=algo-1, epoch=181, train binary_f_1.000 <score>=0.792623069848\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 551.6889095306396, \"sum\": 551.6889095306396, \"min\": 551.6889095306396}}, \"EndTime\": 1604871712.436396, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871711.88409}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:52 INFO 139813082523456] #progress_metric: host=algo-1, completed 91 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 16563, \"sum\": 16563.0, \"min\": 16563}, \"Total Records Seen\": {\"count\": 1, \"max\": 16484740, \"sum\": 16484740.0, \"min\": 16484740}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 183, \"sum\": 183.0, \"min\": 183}}, \"EndTime\": 1604871712.436541, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 181}, \"StartTime\": 1604871711.884682}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:52 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=164093.650337 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:52 INFO 139813082523456] #quality_metric: host=algo-1, epoch=182, batch=0 train binary_classification_accuracy <score>=0.738\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:52 INFO 139813082523456] #quality_metric: host=algo-1, epoch=182, batch=0 train binary_classification_cross_entropy <loss>=0.510117980957\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:52 INFO 139813082523456] #quality_metric: host=algo-1, epoch=182, batch=0 train binary_f_1.000 <score>=0.765652951699\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:52.992] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 366, \"duration\": 554, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:52 INFO 139813082523456] #quality_metric: host=algo-1, epoch=182, train binary_classification_accuracy <score>=0.764978021978\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:52 INFO 139813082523456] #quality_metric: host=algo-1, epoch=182, train binary_classification_cross_entropy <loss>=0.487694450965\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:52 INFO 139813082523456] #quality_metric: host=algo-1, epoch=182, train binary_f_1.000 <score>=0.792932177954\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 555.8509826660156, \"sum\": 555.8509826660156, \"min\": 555.8509826660156}}, \"EndTime\": 1604871712.992908, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871712.43645}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:52 INFO 139813082523456] #progress_metric: host=algo-1, completed 91 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 16654, \"sum\": 16654.0, \"min\": 16654}, \"Total Records Seen\": {\"count\": 1, \"max\": 16575310, \"sum\": 16575310.0, \"min\": 16575310}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 184, \"sum\": 184.0, \"min\": 184}}, \"EndTime\": 1604871712.993046, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 182}, \"StartTime\": 1604871712.437034}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:52 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=162858.742138 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:53 INFO 139813082523456] #quality_metric: host=algo-1, epoch=183, batch=0 train binary_classification_accuracy <score>=0.738\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:53 INFO 139813082523456] #quality_metric: host=algo-1, epoch=183, batch=0 train binary_classification_cross_entropy <loss>=0.509691925049\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:53 INFO 139813082523456] #quality_metric: host=algo-1, epoch=183, batch=0 train binary_f_1.000 <score>=0.765652951699\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:53.535] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 368, \"duration\": 540, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:53 INFO 139813082523456] #quality_metric: host=algo-1, epoch=183, train binary_classification_accuracy <score>=0.765186813187\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:53 INFO 139813082523456] #quality_metric: host=algo-1, epoch=183, train binary_classification_cross_entropy <loss>=0.487319332584\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:53 INFO 139813082523456] #quality_metric: host=algo-1, epoch=183, train binary_f_1.000 <score>=0.793106119287\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 541.9750213623047, \"sum\": 541.9750213623047, \"min\": 541.9750213623047}}, \"EndTime\": 1604871713.535517, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871712.992954}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:53 INFO 139813082523456] #progress_metric: host=algo-1, completed 92 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 16745, \"sum\": 16745.0, \"min\": 16745}, \"Total Records Seen\": {\"count\": 1, \"max\": 16665880, \"sum\": 16665880.0, \"min\": 16665880}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 185, \"sum\": 185.0, \"min\": 185}}, \"EndTime\": 1604871713.535664, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 183}, \"StartTime\": 1604871712.993517}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:53 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=167032.987851 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:53 INFO 139813082523456] #quality_metric: host=algo-1, epoch=184, batch=0 train binary_classification_accuracy <score>=0.738\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:53 INFO 139813082523456] #quality_metric: host=algo-1, epoch=184, batch=0 train binary_classification_cross_entropy <loss>=0.509263122559\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:53 INFO 139813082523456] #quality_metric: host=algo-1, epoch=184, batch=0 train binary_f_1.000 <score>=0.765652951699\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:54.108] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 370, \"duration\": 571, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:54 INFO 139813082523456] #quality_metric: host=algo-1, epoch=184, train binary_classification_accuracy <score>=0.765472527473\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:54 INFO 139813082523456] #quality_metric: host=algo-1, epoch=184, train binary_classification_cross_entropy <loss>=0.486942344917\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:54 INFO 139813082523456] #quality_metric: host=algo-1, epoch=184, train binary_f_1.000 <score>=0.79334185452\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 572.7629661560059, \"sum\": 572.7629661560059, \"min\": 572.7629661560059}}, \"EndTime\": 1604871714.108959, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871713.535572}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:54 INFO 139813082523456] #progress_metric: host=algo-1, completed 92 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 16836, \"sum\": 16836.0, \"min\": 16836}, \"Total Records Seen\": {\"count\": 1, \"max\": 16756450, \"sum\": 16756450.0, \"min\": 16756450}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 186, \"sum\": 186.0, \"min\": 186}}, \"EndTime\": 1604871714.109125, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 184}, \"StartTime\": 1604871713.536171}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:54 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=158050.673691 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:54 INFO 139813082523456] #quality_metric: host=algo-1, epoch=185, batch=0 train binary_classification_accuracy <score>=0.738\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:54 INFO 139813082523456] #quality_metric: host=algo-1, epoch=185, batch=0 train binary_classification_cross_entropy <loss>=0.508831512451\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:54 INFO 139813082523456] #quality_metric: host=algo-1, epoch=185, batch=0 train binary_f_1.000 <score>=0.765652951699\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:54.648] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 372, \"duration\": 537, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:54 INFO 139813082523456] #quality_metric: host=algo-1, epoch=185, train binary_classification_accuracy <score>=0.765703296703\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:54 INFO 139813082523456] #quality_metric: host=algo-1, epoch=185, train binary_classification_cross_entropy <loss>=0.486563487629\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:54 INFO 139813082523456] #quality_metric: host=algo-1, epoch=185, train binary_f_1.000 <score>=0.793559194028\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 539.3619537353516, \"sum\": 539.3619537353516, \"min\": 539.3619537353516}}, \"EndTime\": 1604871714.648991, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871714.109014}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:54 INFO 139813082523456] #progress_metric: host=algo-1, completed 93 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 16927, \"sum\": 16927.0, \"min\": 16927}, \"Total Records Seen\": {\"count\": 1, \"max\": 16847020, \"sum\": 16847020.0, \"min\": 16847020}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 187, \"sum\": 187.0, \"min\": 187}}, \"EndTime\": 1604871714.649199, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 185}, \"StartTime\": 1604871714.1096}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:54 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=167811.440143 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:54 INFO 139813082523456] #quality_metric: host=algo-1, epoch=186, batch=0 train binary_classification_accuracy <score>=0.739\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:54 INFO 139813082523456] #quality_metric: host=algo-1, epoch=186, batch=0 train binary_classification_cross_entropy <loss>=0.508397216797\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:54 INFO 139813082523456] #quality_metric: host=algo-1, epoch=186, batch=0 train binary_f_1.000 <score>=0.766756032172\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:55.221] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 374, \"duration\": 569, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:55 INFO 139813082523456] #quality_metric: host=algo-1, epoch=186, train binary_classification_accuracy <score>=0.765923076923\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:55 INFO 139813082523456] #quality_metric: host=algo-1, epoch=186, train binary_classification_cross_entropy <loss>=0.486182732551\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:55 INFO 139813082523456] #quality_metric: host=algo-1, epoch=186, train binary_f_1.000 <score>=0.793744855967\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 571.8469619750977, \"sum\": 571.8469619750977, \"min\": 571.8469619750977}}, \"EndTime\": 1604871715.22171, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871714.649059}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:55 INFO 139813082523456] #progress_metric: host=algo-1, completed 93 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 17018, \"sum\": 17018.0, \"min\": 17018}, \"Total Records Seen\": {\"count\": 1, \"max\": 16937590, \"sum\": 16937590.0, \"min\": 16937590}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 188, \"sum\": 188.0, \"min\": 188}}, \"EndTime\": 1604871715.221865, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 186}, \"StartTime\": 1604871714.649835}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:55 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=158306.491057 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:55 INFO 139813082523456] #quality_metric: host=algo-1, epoch=187, batch=0 train binary_classification_accuracy <score>=0.739\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:55 INFO 139813082523456] #quality_metric: host=algo-1, epoch=187, batch=0 train binary_classification_cross_entropy <loss>=0.507960144043\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:55 INFO 139813082523456] #quality_metric: host=algo-1, epoch=187, batch=0 train binary_f_1.000 <score>=0.766756032172\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:55.771] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 376, \"duration\": 548, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:55 INFO 139813082523456] #quality_metric: host=algo-1, epoch=187, train binary_classification_accuracy <score>=0.766406593407\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:55 INFO 139813082523456] #quality_metric: host=algo-1, epoch=187, train binary_classification_cross_entropy <loss>=0.485800071632\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:55 INFO 139813082523456] #quality_metric: host=algo-1, epoch=187, train binary_f_1.000 <score>=0.794178874699\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 549.6020317077637, \"sum\": 549.6020317077637, \"min\": 549.6020317077637}}, \"EndTime\": 1604871715.771954, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871715.22176}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:55 INFO 139813082523456] #progress_metric: host=algo-1, completed 94 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 17109, \"sum\": 17109.0, \"min\": 17109}, \"Total Records Seen\": {\"count\": 1, \"max\": 17028160, \"sum\": 17028160.0, \"min\": 17028160}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 189, \"sum\": 189.0, \"min\": 189}}, \"EndTime\": 1604871715.77213, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 187}, \"StartTime\": 1604871715.222326}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:55 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=164705.370372 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:55 INFO 139813082523456] #quality_metric: host=algo-1, epoch=188, batch=0 train binary_classification_accuracy <score>=0.741\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:55 INFO 139813082523456] #quality_metric: host=algo-1, epoch=188, batch=0 train binary_classification_cross_entropy <loss>=0.507520294189\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:55 INFO 139813082523456] #quality_metric: host=algo-1, epoch=188, batch=0 train binary_f_1.000 <score>=0.76854334227\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:56.346] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 378, \"duration\": 572, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:56 INFO 139813082523456] #quality_metric: host=algo-1, epoch=188, train binary_classification_accuracy <score>=0.766582417582\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:56 INFO 139813082523456] #quality_metric: host=algo-1, epoch=188, train binary_classification_cross_entropy <loss>=0.4854154814\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:56 INFO 139813082523456] #quality_metric: host=algo-1, epoch=188, train binary_f_1.000 <score>=0.794333794866\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 574.3210315704346, \"sum\": 574.3210315704346, \"min\": 574.3210315704346}}, \"EndTime\": 1604871716.347003, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871715.772027}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:56 INFO 139813082523456] #progress_metric: host=algo-1, completed 94 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 17200, \"sum\": 17200.0, \"min\": 17200}, \"Total Records Seen\": {\"count\": 1, \"max\": 17118730, \"sum\": 17118730.0, \"min\": 17118730}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 190, \"sum\": 190.0, \"min\": 190}}, \"EndTime\": 1604871716.347207, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 188}, \"StartTime\": 1604871715.772653}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:56 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=157610.339104 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:56 INFO 139813082523456] #quality_metric: host=algo-1, epoch=189, batch=0 train binary_classification_accuracy <score>=0.742\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:56 INFO 139813082523456] #quality_metric: host=algo-1, epoch=189, batch=0 train binary_classification_cross_entropy <loss>=0.507077667236\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:56 INFO 139813082523456] #quality_metric: host=algo-1, epoch=189, batch=0 train binary_f_1.000 <score>=0.769642857143\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:56.889] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 380, \"duration\": 541, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:56 INFO 139813082523456] #quality_metric: host=algo-1, epoch=189, train binary_classification_accuracy <score>=0.766989010989\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:56 INFO 139813082523456] #quality_metric: host=algo-1, epoch=189, train binary_classification_cross_entropy <loss>=0.485028953133\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:56 INFO 139813082523456] #quality_metric: host=algo-1, epoch=189, train binary_f_1.000 <score>=0.794705962086\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 542.8512096405029, \"sum\": 542.8512096405029, \"min\": 542.8512096405029}}, \"EndTime\": 1604871716.890558, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871716.347061}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:56 INFO 139813082523456] #progress_metric: host=algo-1, completed 95 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 17291, \"sum\": 17291.0, \"min\": 17291}, \"Total Records Seen\": {\"count\": 1, \"max\": 17209300, \"sum\": 17209300.0, \"min\": 17209300}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 191, \"sum\": 191.0, \"min\": 191}}, \"EndTime\": 1604871716.890751, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 189}, \"StartTime\": 1604871716.347677}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:56 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=166744.92144 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:56 INFO 139813082523456] #quality_metric: host=algo-1, epoch=190, batch=0 train binary_classification_accuracy <score>=0.743\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:56 INFO 139813082523456] #quality_metric: host=algo-1, epoch=190, batch=0 train binary_classification_cross_entropy <loss>=0.506632202148\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:56 INFO 139813082523456] #quality_metric: host=algo-1, epoch=190, batch=0 train binary_f_1.000 <score>=0.770740410348\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:57.419] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 382, \"duration\": 526, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:57 INFO 139813082523456] #quality_metric: host=algo-1, epoch=190, train binary_classification_accuracy <score>=0.767285714286\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:57 INFO 139813082523456] #quality_metric: host=algo-1, epoch=190, train binary_classification_cross_entropy <loss>=0.484640473083\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:57 INFO 139813082523456] #quality_metric: host=algo-1, epoch=190, train binary_f_1.000 <score>=0.79494950473\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 528.4390449523926, \"sum\": 528.4390449523926, \"min\": 528.4390449523926}}, \"EndTime\": 1604871717.419694, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871716.890615}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:57 INFO 139813082523456] #progress_metric: host=algo-1, completed 95 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 17382, \"sum\": 17382.0, \"min\": 17382}, \"Total Records Seen\": {\"count\": 1, \"max\": 17299870, \"sum\": 17299870.0, \"min\": 17299870}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 192, \"sum\": 192.0, \"min\": 192}}, \"EndTime\": 1604871717.419842, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 190}, \"StartTime\": 1604871716.89123}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:57 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=171309.18299 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:57 INFO 139813082523456] #quality_metric: host=algo-1, epoch=191, batch=0 train binary_classification_accuracy <score>=0.743\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:57 INFO 139813082523456] #quality_metric: host=algo-1, epoch=191, batch=0 train binary_classification_cross_entropy <loss>=0.506183929443\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:57 INFO 139813082523456] #quality_metric: host=algo-1, epoch=191, batch=0 train binary_f_1.000 <score>=0.770740410348\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:57.973] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 384, \"duration\": 552, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:57 INFO 139813082523456] #quality_metric: host=algo-1, epoch=191, train binary_classification_accuracy <score>=0.767626373626\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:57 INFO 139813082523456] #quality_metric: host=algo-1, epoch=191, train binary_classification_cross_entropy <loss>=0.48425002314\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:57 INFO 139813082523456] #quality_metric: host=algo-1, epoch=191, train binary_f_1.000 <score>=0.795263545176\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 553.9288520812988, \"sum\": 553.9288520812988, \"min\": 553.9288520812988}}, \"EndTime\": 1604871717.974286, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871717.41975}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:57 INFO 139813082523456] #progress_metric: host=algo-1, completed 96 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 17473, \"sum\": 17473.0, \"min\": 17473}, \"Total Records Seen\": {\"count\": 1, \"max\": 17390440, \"sum\": 17390440.0, \"min\": 17390440}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 193, \"sum\": 193.0, \"min\": 193}}, \"EndTime\": 1604871717.974434, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 191}, \"StartTime\": 1604871717.420333}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:57 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=163429.767735 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:57 INFO 139813082523456] #quality_metric: host=algo-1, epoch=192, batch=0 train binary_classification_accuracy <score>=0.743\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:57 INFO 139813082523456] #quality_metric: host=algo-1, epoch=192, batch=0 train binary_classification_cross_entropy <loss>=0.505732788086\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:57 INFO 139813082523456] #quality_metric: host=algo-1, epoch=192, batch=0 train binary_f_1.000 <score>=0.770740410348\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[34m[2020-11-08 21:41:58.532] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 386, \"duration\": 556, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:58 INFO 139813082523456] #quality_metric: host=algo-1, epoch=192, train binary_classification_accuracy <score>=0.767879120879\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:58 INFO 139813082523456] #quality_metric: host=algo-1, epoch=192, train binary_classification_cross_entropy <loss>=0.483857586871\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:58 INFO 139813082523456] #quality_metric: host=algo-1, epoch=192, train binary_f_1.000 <score>=0.795480291632\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 558.3081245422363, \"sum\": 558.3081245422363, \"min\": 558.3081245422363}}, \"EndTime\": 1604871718.533269, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871717.974342}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:58 INFO 139813082523456] #progress_metric: host=algo-1, completed 96 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 17564, \"sum\": 17564.0, \"min\": 17564}, \"Total Records Seen\": {\"count\": 1, \"max\": 17481010, \"sum\": 17481010.0, \"min\": 17481010}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 194, \"sum\": 194.0, \"min\": 194}}, \"EndTime\": 1604871718.533413, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 192}, \"StartTime\": 1604871717.974936}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:58 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=162148.851439 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:58 INFO 139813082523456] #quality_metric: host=algo-1, epoch=193, batch=0 train binary_classification_accuracy <score>=0.745\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:58 INFO 139813082523456] #quality_metric: host=algo-1, epoch=193, batch=0 train binary_classification_cross_entropy <loss>=0.505278869629\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:58 INFO 139813082523456] #quality_metric: host=algo-1, epoch=193, batch=0 train binary_f_1.000 <score>=0.772524531668\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:59.119] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 388, \"duration\": 584, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:59 INFO 139813082523456] #quality_metric: host=algo-1, epoch=193, train binary_classification_accuracy <score>=0.768120879121\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:59 INFO 139813082523456] #quality_metric: host=algo-1, epoch=193, train binary_classification_cross_entropy <loss>=0.483463157235\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:59 INFO 139813082523456] #quality_metric: host=algo-1, epoch=193, train binary_f_1.000 <score>=0.795709126819\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 585.8011245727539, \"sum\": 585.8011245727539, \"min\": 585.8011245727539}}, \"EndTime\": 1604871719.119738, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871718.53332}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:59 INFO 139813082523456] #progress_metric: host=algo-1, completed 97 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 17655, \"sum\": 17655.0, \"min\": 17655}, \"Total Records Seen\": {\"count\": 1, \"max\": 17571580, \"sum\": 17571580.0, \"min\": 17571580}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 195, \"sum\": 195.0, \"min\": 195}}, \"EndTime\": 1604871719.119929, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 193}, \"StartTime\": 1604871718.533912}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:59 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=154527.591122 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:59 INFO 139813082523456] #quality_metric: host=algo-1, epoch=194, batch=0 train binary_classification_accuracy <score>=0.745\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:59 INFO 139813082523456] #quality_metric: host=algo-1, epoch=194, batch=0 train binary_classification_cross_entropy <loss>=0.504822052002\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:59 INFO 139813082523456] #quality_metric: host=algo-1, epoch=194, batch=0 train binary_f_1.000 <score>=0.772524531668\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:41:59.676] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 390, \"duration\": 554, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:59 INFO 139813082523456] #quality_metric: host=algo-1, epoch=194, train binary_classification_accuracy <score>=0.768285714286\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:59 INFO 139813082523456] #quality_metric: host=algo-1, epoch=194, train binary_classification_cross_entropy <loss>=0.48306671109\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:59 INFO 139813082523456] #quality_metric: host=algo-1, epoch=194, train binary_f_1.000 <score>=0.795856326847\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 556.5400123596191, \"sum\": 556.5400123596191, \"min\": 556.5400123596191}}, \"EndTime\": 1604871719.676961, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871719.119795}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:59 INFO 139813082523456] #progress_metric: host=algo-1, completed 97 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 17746, \"sum\": 17746.0, \"min\": 17746}, \"Total Records Seen\": {\"count\": 1, \"max\": 17662150, \"sum\": 17662150.0, \"min\": 17662150}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 196, \"sum\": 196.0, \"min\": 196}}, \"EndTime\": 1604871719.677204, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 194}, \"StartTime\": 1604871719.120397}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:59 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=162631.517261 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:59 INFO 139813082523456] #quality_metric: host=algo-1, epoch=195, batch=0 train binary_classification_accuracy <score>=0.745\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:59 INFO 139813082523456] #quality_metric: host=algo-1, epoch=195, batch=0 train binary_classification_cross_entropy <loss>=0.504362365723\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:41:59 INFO 139813082523456] #quality_metric: host=algo-1, epoch=195, batch=0 train binary_f_1.000 <score>=0.772524531668\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:42:00.227] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 392, \"duration\": 548, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:42:00 INFO 139813082523456] #quality_metric: host=algo-1, epoch=195, train binary_classification_accuracy <score>=0.768714285714\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:42:00 INFO 139813082523456] #quality_metric: host=algo-1, epoch=195, train binary_classification_cross_entropy <loss>=0.482668244079\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:42:00 INFO 139813082523456] #quality_metric: host=algo-1, epoch=195, train binary_f_1.000 <score>=0.796208255468\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 558.0718517303467, \"sum\": 558.0718517303467, \"min\": 558.0718517303467}}, \"EndTime\": 1604871720.235833, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871719.677013}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:42:00 INFO 139813082523456] #progress_metric: host=algo-1, completed 98 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 17837, \"sum\": 17837.0, \"min\": 17837}, \"Total Records Seen\": {\"count\": 1, \"max\": 17752720, \"sum\": 17752720.0, \"min\": 17752720}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 197, \"sum\": 197.0, \"min\": 197}}, \"EndTime\": 1604871720.236065, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 195}, \"StartTime\": 1604871719.67773}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:42:00 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=162182.218647 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:42:00 INFO 139813082523456] #quality_metric: host=algo-1, epoch=196, batch=0 train binary_classification_accuracy <score>=0.745\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:42:00 INFO 139813082523456] #quality_metric: host=algo-1, epoch=196, batch=0 train binary_classification_cross_entropy <loss>=0.503899810791\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:42:00 INFO 139813082523456] #quality_metric: host=algo-1, epoch=196, batch=0 train binary_f_1.000 <score>=0.772524531668\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:42:00.781] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 394, \"duration\": 541, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:42:00 INFO 139813082523456] #quality_metric: host=algo-1, epoch=196, train binary_classification_accuracy <score>=0.768978021978\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:42:00 INFO 139813082523456] #quality_metric: host=algo-1, epoch=196, train binary_classification_cross_entropy <loss>=0.482267731048\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:42:00 INFO 139813082523456] #quality_metric: host=algo-1, epoch=196, train binary_f_1.000 <score>=0.796420928273\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 545.1791286468506, \"sum\": 545.1791286468506, \"min\": 545.1791286468506}}, \"EndTime\": 1604871720.781889, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871720.235899}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:42:00 INFO 139813082523456] #progress_metric: host=algo-1, completed 98 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 17928, \"sum\": 17928.0, \"min\": 17928}, \"Total Records Seen\": {\"count\": 1, \"max\": 17843290, \"sum\": 17843290.0, \"min\": 17843290}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 198, \"sum\": 198.0, \"min\": 198}}, \"EndTime\": 1604871720.782123, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 196}, \"StartTime\": 1604871720.236679}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:42:00 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=166011.416195 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:42:00 INFO 139813082523456] #quality_metric: host=algo-1, epoch=197, batch=0 train binary_classification_accuracy <score>=0.745\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:42:00 INFO 139813082523456] #quality_metric: host=algo-1, epoch=197, batch=0 train binary_classification_cross_entropy <loss>=0.503434387207\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:42:00 INFO 139813082523456] #quality_metric: host=algo-1, epoch=197, batch=0 train binary_f_1.000 <score>=0.772524531668\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:42:01.340] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 396, \"duration\": 556, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:42:01 INFO 139813082523456] #quality_metric: host=algo-1, epoch=197, train binary_classification_accuracy <score>=0.769285714286\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:42:01 INFO 139813082523456] #quality_metric: host=algo-1, epoch=197, train binary_classification_cross_entropy <loss>=0.481865169316\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:42:01 INFO 139813082523456] #quality_metric: host=algo-1, epoch=197, train binary_f_1.000 <score>=0.796699944805\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 558.1212043762207, \"sum\": 558.1212043762207, \"min\": 558.1212043762207}}, \"EndTime\": 1604871721.340766, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871720.781954}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:42:01 INFO 139813082523456] #progress_metric: host=algo-1, completed 99 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 18019, \"sum\": 18019.0, \"min\": 18019}, \"Total Records Seen\": {\"count\": 1, \"max\": 17933860, \"sum\": 17933860.0, \"min\": 17933860}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 199, \"sum\": 199.0, \"min\": 199}}, \"EndTime\": 1604871721.340974, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 197}, \"StartTime\": 1604871720.782615}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:42:01 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=162180.418406 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:42:01 INFO 139813082523456] #quality_metric: host=algo-1, epoch=198, batch=0 train binary_classification_accuracy <score>=0.746\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:42:01 INFO 139813082523456] #quality_metric: host=algo-1, epoch=198, batch=0 train binary_classification_cross_entropy <loss>=0.502966033936\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:42:01 INFO 139813082523456] #quality_metric: host=algo-1, epoch=198, batch=0 train binary_f_1.000 <score>=0.773214285714\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:42:01.916] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 398, \"duration\": 573, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:42:01 INFO 139813082523456] #quality_metric: host=algo-1, epoch=198, train binary_classification_accuracy <score>=0.769659340659\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:42:01 INFO 139813082523456] #quality_metric: host=algo-1, epoch=198, train binary_classification_cross_entropy <loss>=0.481460543454\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:42:01 INFO 139813082523456] #quality_metric: host=algo-1, epoch=198, train binary_f_1.000 <score>=0.797040967495\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 575.6950378417969, \"sum\": 575.6950378417969, \"min\": 575.6950378417969}}, \"EndTime\": 1604871721.917154, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871721.340831}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:42:01 INFO 139813082523456] #progress_metric: host=algo-1, completed 99 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 18110, \"sum\": 18110.0, \"min\": 18110}, \"Total Records Seen\": {\"count\": 1, \"max\": 18024430, \"sum\": 18024430.0, \"min\": 18024430}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 200, \"sum\": 200.0, \"min\": 200}}, \"EndTime\": 1604871721.917421, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 198}, \"StartTime\": 1604871721.341431}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:42:01 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=157207.119497 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:42:01 INFO 139813082523456] #quality_metric: host=algo-1, epoch=199, batch=0 train binary_classification_accuracy <score>=0.747\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:42:01 INFO 139813082523456] #quality_metric: host=algo-1, epoch=199, batch=0 train binary_classification_cross_entropy <loss>=0.502494750977\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:42:01 INFO 139813082523456] #quality_metric: host=algo-1, epoch=199, batch=0 train binary_f_1.000 <score>=0.774308652988\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:42:02.504] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 400, \"duration\": 585, \"num_examples\": 91, \"num_bytes\": 5796480}\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:42:02 INFO 139813082523456] #quality_metric: host=algo-1, epoch=199, train binary_classification_accuracy <score>=0.769978021978\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:42:02 INFO 139813082523456] #quality_metric: host=algo-1, epoch=199, train binary_classification_cross_entropy <loss>=0.481053831666\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:42:02 INFO 139813082523456] #quality_metric: host=algo-1, epoch=199, train binary_f_1.000 <score>=0.797315878149\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:42:02 INFO 139813082523456] #quality_metric: host=algo-1, train binary_classification_accuracy <score>=0.769978021978\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:42:02 INFO 139813082523456] #quality_metric: host=algo-1, train binary_classification_cross_entropy <loss>=0.481053831666\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:42:02 INFO 139813082523456] #quality_metric: host=algo-1, train binary_f_1.000 <score>=0.797315878149\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"update.time\": {\"count\": 1, \"max\": 587.0380401611328, \"sum\": 587.0380401611328, \"min\": 587.0380401611328}}, \"EndTime\": 1604871722.505046, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871721.917239}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:42:02 INFO 139813082523456] #progress_metric: host=algo-1, completed 100 % of epochs\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 91, \"sum\": 91.0, \"min\": 91}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Total Batches Seen\": {\"count\": 1, \"max\": 18201, \"sum\": 18201.0, \"min\": 18201}, \"Total Records Seen\": {\"count\": 1, \"max\": 18115000, \"sum\": 18115000.0, \"min\": 18115000}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 90570, \"sum\": 90570.0, \"min\": 90570}, \"Reset Count\": {\"count\": 1, \"max\": 201, \"sum\": 201.0, \"min\": 201}}, \"EndTime\": 1604871722.505192, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\", \"epoch\": 199}, \"StartTime\": 1604871721.917984}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:42:02 INFO 139813082523456] #throughput_metric: host=algo-1, train throughput=154216.563253 records/second\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:42:02 WARNING 139813082523456] wait_for_all_workers will not sync workers since the kv store is not running distributed\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:42:02 INFO 139813082523456] Pulling entire model from kvstore to finalize\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"finalize.time\": {\"count\": 1, \"max\": 1.6951560974121094, \"sum\": 1.6951560974121094, \"min\": 1.6951560974121094}}, \"EndTime\": 1604871722.507077, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871722.505101}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:42:02 INFO 139813082523456] Saved checkpoint to \"/tmp/tmpMY8i6n/state-0001.params\"\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:42:02.512] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/test\", \"epoch\": 0, \"duration\": 113115, \"num_examples\": 1, \"num_bytes\": 64000}\u001b[0m\n",
      "\u001b[34m[2020-11-08 21:42:02.546] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/test\", \"epoch\": 1, \"duration\": 33, \"num_examples\": 10, \"num_bytes\": 603520}\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 10, \"sum\": 10.0, \"min\": 10}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 10, \"sum\": 10.0, \"min\": 10}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 9430, \"sum\": 9430.0, \"min\": 9430}, \"Total Batches Seen\": {\"count\": 1, \"max\": 10, \"sum\": 10.0, \"min\": 10}, \"Total Records Seen\": {\"count\": 1, \"max\": 9430, \"sum\": 9430.0, \"min\": 9430}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 9430, \"sum\": 9430.0, \"min\": 9430}, \"Reset Count\": {\"count\": 1, \"max\": 1, \"sum\": 1.0, \"min\": 1}}, \"EndTime\": 1604871722.5464, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"test_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871722.512647}\n",
      "\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:42:02 INFO 139813082523456] #test_score (algo-1) : ('binary_classification_accuracy', 0.699575821845175)\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:42:02 INFO 139813082523456] #test_score (algo-1) : ('binary_classification_cross_entropy', 0.573116159287389)\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:42:02 INFO 139813082523456] #test_score (algo-1) : ('binary_f_1.000', 0.7423842866236247)\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:42:02 INFO 139813082523456] #quality_metric: host=algo-1, test binary_classification_accuracy <score>=0.699575821845\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:42:02 INFO 139813082523456] #quality_metric: host=algo-1, test binary_classification_cross_entropy <loss>=0.573116159287\u001b[0m\n",
      "\u001b[34m[11/08/2020 21:42:02 INFO 139813082523456] #quality_metric: host=algo-1, test binary_f_1.000 <score>=0.742384286624\u001b[0m\n",
      "\u001b[34m#metrics {\"Metrics\": {\"totaltime\": {\"count\": 1, \"max\": 113199.23496246338, \"sum\": 113199.23496246338, \"min\": 113199.23496246338}, \"setuptime\": {\"count\": 1, \"max\": 40.16304016113281, \"sum\": 40.16304016113281, \"min\": 40.16304016113281}}, \"EndTime\": 1604871722.547167, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"factorization-machines\"}, \"StartTime\": 1604871722.507131}\n",
      "\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "2020-11-08 21:42:12 Uploading - Uploading generated training model\n",
      "2020-11-08 21:42:12 Completed - Training job completed\n",
      "Training seconds: 164\n",
      "Billable seconds: 164\n"
     ]
    }
   ],
   "source": [
    "fm.set_hyperparameters(feature_dim=nb_features,\n",
    "                       predictor_type='binary_classifier',\n",
    "                       mini_batch_size=1000,\n",
    "                       num_factors=64,\n",
    "                       epochs=200)\n",
    "fm.fit({'train': fm_train_data_path, 'test': fm_test_data_path})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "import json\n",
    "#from sagemaker.predictor import json_deserializer\n",
    "\n",
    "def fm_serializer(data):\n",
    "    js = {'instances': []}\n",
    "    for row in data:\n",
    "        js['instances'].append({'features': row.tolist()})\n",
    "    #print(json.dumps(js))\n",
    "    return json.dumps(js)\n",
    "\n",
    "# fm_predictor.content_type = 'application/json'\n",
    "# fm_predictor.serializer = fm_serializer\n",
    "# fm_predictor.deserializer = json_deserializer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "-----------------"
     ]
    }
   ],
   "source": [
    "fm_predictor = fm.deploy(initial_instance_count=1,\n",
    "                         instance_type='ml.m5.xlarge',                         \n",
    "                         serializer=fm_serializer,\n",
    "                         deserializer=sagemaker.deserializers.JSONDeserializer())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 2 - Extracting parameters from FM model\n",
    "\n",
    "Now that we have the model created and stored in SageMaker, we can download the same and extract the parameters.  The FM model is stored in MxNet format.\n",
    "\n",
    "This section is reproduced with minor modifications from the blog cited above for the sake of completeness."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Download model data\n",
    "\n",
    "Skip the next cell block if you have already downloaded the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import mxnet as mx\n",
    "model_file_name = \"model.tar.gz\"\n",
    "model_full_path = fm.output_path +\"/\"+ fm.latest_training_job.job_name +\"/output/\"+model_file_name\n",
    "print(\"Model Path: \", model_full_path)\n",
    "\n",
    "#Download FM model \n",
    "os.system(\"aws s3 cp \"+model_full_path+ \" .\")\n",
    "\n",
    "#Extract model file for loading to MXNet\n",
    "os.system(\"tar xzvf \"+model_file_name)\n",
    "os.system(\"unzip -o model_algo-1\")\n",
    "os.system(\"mv symbol.json model-symbol.json\")\n",
    "os.system(\"mv params model-0000.params\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Extract model data to create item and user latent matrixes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import mxnet as mx\n",
    "#Extract model data\n",
    "m = mx.module.Module.load('./model', 0, False, label_names=['out_label'])\n",
    "V = m._arg_params['v'].asnumpy()\n",
    "w = m._arg_params['w1_weight'].asnumpy()\n",
    "b = m._arg_params['w0_weight'].asnumpy()\n",
    "\n",
    "# item latent matrix - concat(V[i], w[i]).  \n",
    "knn_item_matrix = np.concatenate((V[nb_users:], w[nb_users:]), axis=1)\n",
    "knn_train_label = np.arange(1,nb_movies+1)\n",
    "\n",
    "#user latent matrix - concat (V[u], 1) \n",
    "ones = np.ones(nb_users).reshape((nb_users, 1))\n",
    "knn_user_matrix = np.concatenate((V[:nb_users], ones), axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 3: Calculate Influence Matrix\n",
    "\n",
    "Per the paper cited above, the influence matrix for user $j$ is calculated as:\n",
    "\n",
    "$$J_j=U^T(U W_j U^T)^{-1}UW_j$$\n",
    "\n",
    "Let's map those symbols to the variables in this notebook.\n",
    "\n",
    "* $U$ is the embedding matrix for items.  In this formula, it is the transpose of the item matrix we extracted from the FM model.  So $U={knn\\_item\\_matrix}^{T}$\n",
    "* $U^T={knn\\_item\\_matrix}$\n",
    "* $W$ is a binary matrix with 1s on the diagonal in positions corresponding the known entries of X for this user.  In other words, it's a matrix of size $nb\\_movies$ by $nb\\_movies$, with a one on the diagonal in row and column $i$ where user $j$ rated movie $i$.\n",
    "\n",
    "Now let's confirm that our dimensions line up properly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "knn_item_matrix.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "knn_user_matrix.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Build the matrix $W$.\n",
    "\n",
    "For the sake of an example, let's pick user `846`, just because that user was the first row in our training set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "W = np.zeros([nb_movies,nb_movies])\n",
    "W.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "user_of_interest = 846\n",
    "\n",
    "u1 = user_movie_ratings_train[user_movie_ratings_train.user_id == user_of_interest]\n",
    "u2 = user_movie_ratings_test[user_movie_ratings_test.user_id == user_of_interest]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "u1.head(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "u1 = u1[u1.rating >= 4] # we only include ratings of 4 or more\n",
    "u2 = u2[u2.rating >= 4]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "u_all = np.concatenate((np.array(u1['movie_id']), np.array(u2['movie_id'])), axis=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for u_rating in u_all:\n",
    "    W[u_rating,u_rating] = 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Calculate $J$ for user $j$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# influence matrix = u_tr * (u*w*u_tr)-1 * u * w\n",
    "J1 = np.matmul(np.transpose(knn_item_matrix), W) # u*w\n",
    "J2 = np.matmul(J1, knn_item_matrix) # u*w*u_tr\n",
    "J3 = np.linalg.inv(J2) # (u*w*u_tr)-1\n",
    "J4 = np.matmul(knn_item_matrix, J3) # u_tr * (u*w*u_tr)-1\n",
    "J5 = np.matmul(J4, np.transpose(knn_item_matrix)) # u_tr * (u*w*u_tr)-1 * u\n",
    "J = np.matmul(J5, W) # # u_tr * (u*w*u_tr)-1 * u * w"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "J.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 4: Explaining recommendations for a user\n",
    "\n",
    "Now we can use the influence matrix to calculate the two metrics explained in the research paper:\n",
    "\n",
    "_Influence_ of the actual rating that user $j$ assigned to item $k$ on the predicted rating for item $i$.  This is calculated as:\n",
    "\n",
    "$${\\beta}_k = J_{ik}^j$$\n",
    "\n",
    "In other words, we just look up the element at row $i$ and column $k$ of the influence matrix $J$ for user $j$\n",
    "\n",
    "_Impact_ of the actual rating that user $j$ assigned to item $k$ on the predicted rating for item $i$.  This is calculated as:\n",
    "\n",
    "$${\\gamma}_k = {\\beta}_{k}x_{kj}$$\n",
    "\n",
    "In other words, we multiply the influence by the actual rating that user $j$ gave to item $k$\n",
    "\n",
    "In this example I'll just use influence, since we converted the ratings to a binary like/don't like.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Look up influence for a test recommendation\n",
    "\n",
    "For our selected user, let's find a movie in our test set that they rated."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "u2.head(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "movie_to_rate = 60"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "result = fm_predictor.predict(X_test[8451:8452].toarray()) # use the row number from the test set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "result"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For movie 60, the user provided a rating of 4, and the FM model predicted that they'd like it with a score of 0.75.\n",
    "\n",
    "Let's see what influenced that rating."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "influence_i = J[movie_to_rate-1,:] # movies are indexed at 1, so we offset to 0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "influence_i[movie_to_rate-1] = 0.0 # zero this out; it's the influence of the movie itself"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# join with movie names\n",
    "df_movies = pd.read_csv('ml-100k/u.item', sep='|', header=None, names=['movie_id', 'movie_name', 'c3','c4','c5','c6','c7',\n",
    "                                                                      'c9','c9','c10','c11','c12','c13','c14','c15','c16','c17',\n",
    "                                                                      'c18','c19','c20','c21','c22','c23','c24'])\n",
    "df_movies.head(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_influence = pd.DataFrame(data={'influence': influence_i, 'movie': df_movies['movie_name']})\n",
    "df_influence.head(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This movie is 'Three Colors:Blue', a French drama that probably appeals to 'art house' movie goers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_movies[df_movies['movie_id'] == movie_to_rate]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And what do we recommend?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_top_influence = df_influence.nlargest(20, 'influence')\n",
    "df_top_influence"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "plt.style.use('ggplot')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ax = df_top_influence.plot(x ='movie', y='influence', kind = 'barh', figsize=(20,20), title='Top 20 Influences', color='blue')\n",
    "ax.set_ylabel(\"Movie\")\n",
    "ax.set_xlabel(\"Influence\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "These influences seem to make sense.  The first two are the other movies in the same trilogy, and the others seem to make sense if you're a movie fan.  `Short Cuts`, for example, is an indie film by Robert Altman, which might appeal to the people who liked the 'Three Colors' trilogy.''"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Look up influence for new recommendation\n",
    "\n",
    "Now let's consider a movie that the user hasn't seen before."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "np.sort(u_all)[:5]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "movie_to_rate = 9"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "rate_data = np.zeros((1, nb_features))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "rate_data[0, user_of_interest-1] = 1.0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "rate_data[0, nb_users + movie_to_rate -1] = 1.0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "result = fm_predictor.predict(rate_data) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "result"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The model predicts that the user will like this movie.  Let's see why."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "influence_i = J[movie_to_rate-1,:] # movies are indexed at 1, so we offset to 0\n",
    "influence_i[movie_to_rate-1] = 0.0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_influence = pd.DataFrame(data={'influence': influence_i, 'movie': df_movies['movie_name']})\n",
    "df_influence.head(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We're looking at the movie 'Dead Man Walking', which was an acclaimed movie about a prisoner on Death Row."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_movies[df_movies['movie_id'] == movie_to_rate]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_top_influence = df_influence.nlargest(20, 'influence')\n",
    "df_top_influence"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ax = df_top_influence.plot(x ='movie', y='influence', kind = 'barh', figsize=(20,20), title='Top 20 Influences', color='blue')\n",
    "ax.set_ylabel(\"Movie\")\n",
    "ax.set_xlabel(\"Influence\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Are these results intuitively satisfying?  I'm not quite sure, but remember that built this model with a relatively limited data set."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Release Resources"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%javascript\n",
    "Jupyter.notebook.save_checkpoint();\n",
    "Jupyter.notebook.session.delete();"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "conda_python3",
   "language": "python",
   "name": "conda_python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
